blob: abb42ab5898a500e4b83da1789ee5be400b277ae [file] [log] [blame]
Kailash92764922019-07-25 08:21:39 -07001# Copyright 2017-present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
Kailash92764922019-07-25 08:21:39 -070014# robot test functions
15
16*** Settings ***
17Documentation Library for various utilities
18Library SSHLibrary
Kailash92764922019-07-25 08:21:39 -070019Library String
20Library DateTime
21Library Process
22Library Collections
23Library RequestsLibrary
24Library OperatingSystem
Andy Bavier63460da2020-02-20 14:35:12 -070025Library CORDRobot
26Library ImportResource resources=CORDRobot
Kailash92764922019-07-25 08:21:39 -070027
28*** Keywords ***
Kailash528f7c02019-07-31 10:55:49 -070029Check CLI Tools Configured
30 [Documentation] Tests that use 'voltctl' and 'kubectl' should execute this keyword in suite setup
31 # check voltctl and kubectl configured
Zack Williamsa8fe75a2020-01-10 14:25:27 -070032 ${voltctl_rc}= Run And Return RC voltctl device list
33 ${kubectl_rc}= Run And Return RC kubectl get pods
Gilles Depatie675a2062019-10-22 12:44:42 -040034 Run Keyword If ${voltctl_rc} != 0 or ${kubectl_rc} != 0 FATAL ERROR
35 ... VOLTCTL and KUBECTL not configured. Please configure before executing tests.
Gilles Depatieb5682f82019-10-31 10:39:45 -040036
37Send File To Onos
Zack Williamsa8fe75a2020-01-10 14:25:27 -070038 [Documentation] Send the content of the file to Onos to selected section of configuration
39 ... using Post Request
40 [Arguments] ${CONFIG_FILE} ${section}
41 ${Headers}= Create Dictionary Content-Type application/json
42 ${File_Data}= OperatingSystem.Get File ${CONFIG_FILE}
43 Log ${Headers}
44 Log ${File_Data}
45 ${resp}= Post Request ONOS
46 ... /onos/v1/network/configuration/${section} headers=${Headers} data=${File_Data}
47 Should Be Equal As Strings ${resp.status_code} 200
Andy Bavier88cd9f62019-11-26 16:22:33 -070048
49Common Test Suite Setup
50 [Documentation] Setup the test suite
Andy Bavier88cd9f62019-11-26 16:22:33 -070051 Set Global Variable ${KUBECTL_CONFIG} export KUBECONFIG=%{KUBECONFIG}
52 Set Global Variable ${VOLTCTL_CONFIG} export VOLTCONFIG=%{VOLTCONFIG}
53 ${k8s_node_ip}= Evaluate ${nodes}[0].get("ip")
ubuntu6b6e7d42020-03-02 12:35:42 -080054 ${ONOS_REST_IP}= Get Environment Variable ONOS_REST_IP ${k8s_node_ip}
55 ${ONOS_SSH_IP}= Get Environment Variable ONOS_SSH_IP ${k8s_node_ip}
56 Set Global Variable ${ONOS_REST_IP}
57 Set Global Variable ${ONOS_SSH_IP}
Andy Bavier88cd9f62019-11-26 16:22:33 -070058 ${k8s_node_user}= Evaluate ${nodes}[0].get("user")
59 ${k8s_node_pass}= Evaluate ${nodes}[0].get("pass")
60 Check CLI Tools Configured
61 ${onos_auth}= Create List karaf karaf
62 ${HEADERS} Create Dictionary Content-Type=application/json
ubuntu6b6e7d42020-03-02 12:35:42 -080063 Create Session ONOS http://${ONOS_REST_IP}:${ONOS_REST_PORT} auth=${ONOS_AUTH}
Andy Bavier88cd9f62019-11-26 16:22:33 -070064 ${olt_ip}= Evaluate ${olts}[0].get("ip")
Suchitra Vemurib7253a52020-07-14 22:35:17 -070065 ${olt_ssh_ip}= Evaluate ${olts}[0].get("sship")
Andy Bavier88cd9f62019-11-26 16:22:33 -070066 ${olt_user}= Evaluate ${olts}[0].get("user")
67 ${olt_pass}= Evaluate ${olts}[0].get("pass")
68 ${olt_serial_number}= Evaluate ${olts}[0].get("serial")
69 ${num_onus}= Get Length ${hosts.src}
70 ${num_onus}= Convert to String ${num_onus}
71 #send sadis file to onos
Andy Baviera2691752019-11-27 12:18:38 -070072 ${sadis_file}= Get Variable Value ${sadis.file}
Zack Williamsa8fe75a2020-01-10 14:25:27 -070073 Log To Console \nSadis File:${sadis_file}
Andy Bavier88cd9f62019-11-26 16:22:33 -070074 Run Keyword Unless '${sadis_file}' is '${None}' Send File To Onos ${sadis_file} apps/
75 Set Suite Variable ${num_onus}
76 Set Suite Variable ${olt_serial_number}
77 Set Suite Variable ${olt_ip}
Suchitra Vemurib7253a52020-07-14 22:35:17 -070078 Set Suite Variable ${olt_ssh_ip}
Andy Bavier88cd9f62019-11-26 16:22:33 -070079 Set Suite Variable ${olt_user}
80 Set Suite Variable ${olt_pass}
Andy Bavier88cd9f62019-11-26 16:22:33 -070081 @{container_list}= Create List adapter-open-olt adapter-open-onu voltha-api-server
82 ... voltha-ro-core voltha-rw-core-11 voltha-rw-core-12 voltha-ofagent
83 Set Suite Variable ${container_list}
84 ${datetime}= Get Current Date
suraj gour7f6d5fe2019-11-29 10:56:35 +000085 Set Suite Variable ${datetime}
86
87WPA Reassociate
suraj gour7f6d5fe2019-11-29 10:56:35 +000088 [Documentation] Executes a particular wpa_cli reassociate, which performs force reassociation
Zack Williamsa8fe75a2020-01-10 14:25:27 -070089 [Arguments] ${iface} ${ip} ${user} ${pass}=${None}
90 ... ${container_type}=${None} ${container_name}=${None}
suraj gour7f6d5fe2019-11-29 10:56:35 +000091 #Below for loops are used instead of sleep time, to execute reassociate command and check status
Zack Williamsa8fe75a2020-01-10 14:25:27 -070092 FOR ${i} IN RANGE 70
93 ${output}= Login And Run Command On Remote System
94 ... wpa_cli -i ${iface} reassociate ${ip} ${user}
95 ... ${pass} ${container_type} ${container_name}
96 ${passed}= Run Keyword And Return Status Should Contain ${output} OK
Andy Bavierb63f6d22020-03-12 15:34:37 -070097 Exit For Loop If ${passed}
Zack Williamsa8fe75a2020-01-10 14:25:27 -070098 END
Andy Bavierb63f6d22020-03-12 15:34:37 -070099 Should Be True ${passed} Status does not contain 'SUCCESS'
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700100 FOR ${i} IN RANGE 70
101 ${output}= Login And Run Command On Remote System
Andy Bavierae4a17e2020-02-25 14:54:36 -0700102 ... wpa_cli -i ${iface} status | grep SUCCESS ${ip} ${user}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700103 ... ${pass} ${container_type} ${container_name}
104 ${passed}= Run Keyword And Return Status Should Contain ${output} SUCCESS
Andy Bavierb63f6d22020-03-12 15:34:37 -0700105 Exit For Loop If ${passed}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700106 END
Andy Bavierb63f6d22020-03-12 15:34:37 -0700107 Should Be True ${passed} Status does not contain 'SUCCESS'
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700108
suraj gour7f6d5fe2019-11-29 10:56:35 +0000109Validate Authentication After Reassociate
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700110 [Arguments] ${auth_pass} ${iface} ${ip} ${user} ${pass}=${None}
111 ... ${container_type}=${None} ${container_name}=${None}
112 [Documentation]
113 ... Executes a particular reassociate request on the RG using wpa_cli.
114 ... auth_pass determines if authentication should pass
Andy Bavier84834d42020-02-25 13:49:50 -0700115 ${wpa_log}= Catenate SEPARATOR=. /tmp/wpa ${iface} log
116 ${output}= Login And Run Command On Remote System truncate -s 0 ${wpa_log}; cat ${wpa_log}
117 ... ${ip} ${user} ${pass} ${container_type} ${container_name}
118 Log ${output}
119 Should Not Contain ${output} authentication completed successfully
suraj gour7f6d5fe2019-11-29 10:56:35 +0000120 WPA Reassociate ${iface} ${ip} ${user} ${pass} ${container_type} ${container_name}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700121 Run Keyword If '${auth_pass}' == 'True' Wait Until Keyword Succeeds ${timeout} 2s
Andy Bavier84834d42020-02-25 13:49:50 -0700122 ... Check Remote File Contents True ${wpa_log} ${iface}.*authentication completed successfully
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700123 ... ${ip} ${user} ${pass} ${container_type} ${container_name}
suraj gour7f6d5fe2019-11-29 10:56:35 +0000124 Run Keyword If '${auth_pass}' == 'False' Sleep 20s
Suchitra Vemuri3cb29f82020-02-25 13:59:58 -0800125 Run Keyword If '${auth_pass}' == 'False' Check Remote File Contents False /tmp/wpa.log
Andy Bavier84834d42020-02-25 13:49:50 -0700126 ... ${iface}.*authentication completed successfully ${ip} ${user} ${pass}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700127 ... ${container_type} ${container_name}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000128
129Send Dhclient Request To Release Assigned IP
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700130 [Arguments] ${iface} ${ip} ${user} ${path_dhcpleasefile} ${pass}=${None}
131 ... ${container_type}=${None} ${container_name}=${None}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000132 [Documentation] Executes a dhclient with option to release ip against a particular interface on the RG (src)
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700133 ${result}= Login And Run Command On Remote System
134 ... dhclient -nw -r ${iface} && rm ${path_dhcpleasefile}/dhclient.* ${ip} ${user}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000135 ... ${pass} ${container_type} ${container_name}
136 Log ${result}
137 #Should Contain ${result} DHCPRELEASE
138 [Return] ${result}
suraj gour1ecfae92019-12-20 15:11:40 +0000139
140Check Remote File Contents For WPA Logs
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700141 [Arguments] ${file_should_exist} ${file} ${pattern} ${ip} ${user} ${pass}=${None}
142 ... ${container_type}=${None} ${container_name}=${None} ${prompt}=~$
suraj gour1ecfae92019-12-20 15:11:40 +0000143 [Documentation] Checks for particular pattern count in a file
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700144 ${result}= Login And Run Command On Remote System
145 ... cat ${file} | grep '${pattern}' | wc -l ${ip} ${user} ${pass}
suraj gour1ecfae92019-12-20 15:11:40 +0000146 ... ${container_type} ${container_name} ${prompt}
147 [Return] ${result}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000148
149Perform Sanity Test
150 [Documentation] This keyword performs Sanity Test Procedure
151 ... Sanity test performs authentication, dhcp and pings for all the ONUs given for the POD
152 ... This keyword can be used to call in any other tests where sanity check is required
153 ... and avoids duplication of code.
154 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
155 Set Global Variable ${of_id}
Hardik Windlass21807632020-04-14 16:24:55 +0530156 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
157 ... Get NNI Port in ONOS ${of_id}
158 Set Global Variable ${nni_port}
Hardik Windlass2a9b1592020-07-23 18:28:13 +0530159 # Verify Default Meter in ONOS (valid only for ATT)
160 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
161 ... Verify Default Meter Present in ONOS ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000162 FOR ${I} IN RANGE 0 ${num_onus}
163 ${src}= Set Variable ${hosts.src[${I}]}
164 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuri03005912020-02-20 18:59:46 -0800165 Run Keyword and Ignore Error Collect Logs
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000166 ${onu_device_id}= Get Device ID From SN ${src['onu']}
167 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
168 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800169 # Check ONU port is Enabled in ONOS
170 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700171 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800172 # Verify EAPOL flows are added for the ONU port
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000173 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800174 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800175 # Verify ONU state in voltha
176 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
177 ... ENABLED ACTIVE REACHABLE
178 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
179 # Perform Authentication
Andy Bavier84834d42020-02-25 13:49:50 -0700180 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
181 ... /tmp/wpa ${src['dp_iface_name']} log
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000182 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
183 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
Andy Bavier84834d42020-02-25 13:49:50 -0700184 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000185 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800186 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000187 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800188 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000189 ... volt-add-subscriber-access ${of_id} ${onu_port}
Suchitra Vemuri9da44302020-03-04 14:24:49 -0800190 # Verify that no pending flows exist for the ONU port
191 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
192 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530193 # Verify subscriber access flows are added for the ONU port
194 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
195 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
196 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Hardik Windlass2a9b1592020-07-23 18:28:13 +0530197 # Verify Meters in ONOS
198 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
199 ... Verify Meters in ONOS ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id} ${onu_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000200 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
201 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
202 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
203 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
204 ... ${dst['container_name']}
205 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800206 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000207 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
208 Run Keyword and Ignore Error Collect Logs
209 END
210
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000211Perform Sanity Test DT
212 [Documentation] This keyword performs Sanity Test Procedure for DT Workflow
213 ... Sanity test performs dhcp and pings (without EAPOL and DHCP flows) for all the ONUs given for the POD
214 ... This keyword can be used to call in any other tests where sanity check is required
215 ... and avoids duplication of code.
216 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
217 Set Global Variable ${of_id}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530218 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
219 ... Get NNI Port in ONOS ${of_id}
220 Set Global Variable ${nni_port}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000221 FOR ${I} IN RANGE 0 ${num_onus}
222 ${src}= Set Variable ${hosts.src[${I}]}
223 ${dst}= Set Variable ${hosts.dst[${I}]}
224 ${onu_device_id}= Get Device ID From SN ${src['onu']}
225 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
226 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
227 # Check ONU port is Enabled in ONOS
228 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700229 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000230 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800231 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000232 ... volt-add-subscriber-access ${of_id} ${onu_port}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530233 # Verify subscriber access flows are added for the ONU port
234 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
Hardik Windlass25e11702020-03-30 20:05:19 +0530235 ... Verify Subscriber Access Flows Added For ONU DT ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
236 ... ${onu_port} ${nni_port} ${src['s_tag']}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000237 # Verify ONU state in voltha
238 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
239 ... ENABLED ACTIVE REACHABLE
240 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Hardik Windlass2a9b1592020-07-23 18:28:13 +0530241 # Verify Meters in ONOS
242 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
243 ... Verify Meters in ONOS ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id} ${onu_port}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530244 # TODO: Yet to Verify on the GPON based Physical POD (VOL-2652)
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000245 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
246 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
247 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
248 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
249 ... ${dst['container_name']}
250 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
251 Run Keyword and Ignore Error Collect Logs
252 END
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530253 # Verify ONOS Flows
254 # Number of Access Flows on ONOS equals 4 * the Number of Active ONUs (2 for each downstream and upstream)
255 ${onos_flows_count}= Evaluate 4 * ${num_onus}
256 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
Hardik Windlass25e11702020-03-30 20:05:19 +0530257 ... Verify Subscriber Access Flows Added Count DT ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
258 ... ${onos_flows_count}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530259 # Verify VOLTHA Flows
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000260 # Number of per OLT Flows equals Twice the Number of Active ONUs (each for downstream and upstream) + 1 for LLDP
261 ${olt_flows}= Evaluate 2 * ${num_onus} + 1
262 # Number of per ONU Flows equals 2 (one each for downstream and upstream)
263 ${onu_flows}= Set Variable 2
264 Run Keyword Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Flows ${olt_flows}
265 ${List_ONU_Serial} Create List
266 Set Suite Variable ${List_ONU_Serial}
267 Build ONU SN List ${List_ONU_Serial}
268 Log ${List_ONU_Serial}
269 Run Keyword Wait Until Keyword Succeeds ${timeout} 5s Validate ONU Flows
270 ... ${List_ONU_Serial} ${onu_flows}
271
Suchitra Vemuri2f1e07d2020-06-26 19:18:01 -0700272Perform Sanity Test TT
273 [Documentation] This keyword performs Sanity Test Procedure for TT Workflow
274 ... Sanity test performs dhcp and pings (without EAPOL and DHCP flows) for all the ONUs given for the POD
275 ... This keyword can be used to call in any other tests where sanity check is required
276 ... and avoids duplication of code.
Suchitra Vemuri85cb7d62020-07-16 23:46:16 -0700277 FOR ${I} IN RANGE 0 ${num_onus}
278 ${src}= Set Variable ${hosts.src[${I}]}
279 ${dst}= Set Variable ${hosts.dst[${I}]}
280 ${service_type}= Get Variable Value ${src['service_type']} "null"
281 Run Keyword IF '${service_type}' != 'mcast' Sanity Test TT one ONU ${src} ${dst}
282 Run Keyword and Ignore Error Collect Logs
283 END
284 Run Keyword and Ignore Error Collect Logs
285
286Sanity Test TT one ONU
287 [Documentation] This keyword performs sanity test for a single ONU for TT workflow
288 ... Tests for one ONU
289 ... Assertions apply to HSIA, VoD, VoIP services
290 [Arguments] ${src} ${dst}
Suchitra Vemuri2f1e07d2020-06-26 19:18:01 -0700291 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
292 Set Global Variable ${of_id}
293 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
294 ... Get NNI Port in ONOS ${of_id}
295 Set Global Variable ${nni_port}
Suchitra Vemuri85cb7d62020-07-16 23:46:16 -0700296 ${onu_device_id}= Get Device ID From SN ${src['onu']}
297 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
298 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
299 # Check ONU port is Enabled in ONOS
300 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
301 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
302 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
303 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
304 ... volt-add-subscriber-access ${of_id} ${onu_port}
305 Sleep 30s
306 # Verify ONU state in voltha
307 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
308 ... ENABLED ACTIVE REACHABLE
309 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
310 # TODO: Yet to Verify on the GPON based Physical POD (VOL-2652)
311 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
312 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
313 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
314 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
315 ... ${dst['container_name']}
Suchitra Vemuri8d3b0032020-09-11 17:49:08 -0700316 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
317 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri85cb7d62020-07-16 23:46:16 -0700318 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
319 Run Keyword and Ignore Error Collect Logs
320
321Perform Sanity Test TT MCAST
322 [Documentation] This keyword performs Sanity Test Procedure for TT Workflow
323 ... Adds subscribers
324 ... Validates for MCAST
Suchitra Vemuri2f1e07d2020-06-26 19:18:01 -0700325 FOR ${I} IN RANGE 0 ${num_onus}
326 ${src}= Set Variable ${hosts.src[${I}]}
327 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuri85cb7d62020-07-16 23:46:16 -0700328 ${service_type}= Get Variable Value ${src['service_type']} "null"
329 Run Keyword IF '${service_type}' == 'mcast' Sanity Test TT MCAST one ONU ${src}
330 ... ${dst}
Suchitra Vemuri2f1e07d2020-06-26 19:18:01 -0700331 Run Keyword and Ignore Error Collect Logs
332 END
Suchitra Vemuri85cb7d62020-07-16 23:46:16 -0700333 Run Keyword and Ignore Error Collect Logs
334
335Sanity Test TT MCAST one ONU
336 [Documentation] This keyword performs sanity test for a single ONU for TT workflow
337 ... Tests for one ONU
338 ... Assertions apply to MCAST services
339 [Arguments] ${src} ${dst}
340 # Check for iperf and jq tools
341 ${stdout} ${stderr} ${rc}= Execute Remote Command which iperf jq
342 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']}
343 ... ${src['container_name']}
344 Pass Execution If ${rc} != 0 Skipping test: iperf / jq not found on the RG
345
346 #Reset the IP on the interface
347 ${output}= Login And Run Command On Remote System sudo ifconfig ${src['dp_iface_name']} 0
348 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
349 # Kill iperf on BNG
350 ${rg_output}= Run Keyword and Continue On Failure Login And Run Command On Remote System
351 ... sudo kill -9 `pidof iperf`
352 ... ${dst['bng_ip']} ${dst['bng_user']} ${dst['bng_pass']} ${dst['container_type']}
353 ... ${dst['container_name']}
354
355 # Setup RG for Multi-cast test
356 ${output}= Login And Run Command On Remote System
357 ... sudo ifconfig ${src['dp_iface_name']} ${src['mcast_rg']} up ; sudo kill -9 `pidof iperf`
358 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
359 ${output}= Login And Run Command On Remote System
360 ... sudo ip route add ${src['mcast_grp_subnet_mask']} dev ${src['dp_iface_name']} scope link
361 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
362
363 # Perform operations for adding subscriber
364 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
365 Set Global Variable ${of_id}
366 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
367 ... Get NNI Port in ONOS ${of_id}
368 Set Global Variable ${nni_port}
369 ${onu_device_id}= Get Device ID From SN ${src['onu']}
370 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
371 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
372 # Check ONU port is Enabled in ONOS
373 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
374 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
375 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
376 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
377 ... volt-add-subscriber-access ${of_id} ${onu_port}
378 Sleep 30s
379 # Verify ONU state in voltha
380 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
381 ... ENABLED ACTIVE REACHABLE
382 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
383
384 # Setup iperf on the BNG
385 ${server_output}= Login And Run Command On Remote System
386 ... sudo iperf -c ${dst['dp_iface_ip_qinq']} -u -T 32 -t 60 -i 1 &
387 ... ${dst['bng_ip']} ${dst['bng_user']} ${dst['bng_pass']} ${dst['container_type']}
388 ... ${dst['container_name']}
389
390 # Setup iperf on the RG
391 ${rg_output}= Run Keyword and Continue On Failure Wait Until Keyword Succeeds 90s 5s
392 ... Login And Run Command On Remote System
393 ... rm -rf /tmp/rg_output ; sudo iperf -s -u -B ${dst['dp_iface_ip_qinq']} -i 1 -D >> /tmp/rg_output
394 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
395 Log ${rg_output}
396 Sleep 60s
397 ${output}= Run Keyword and Continue On Failure Wait Until Keyword Succeeds 90s 5s
398 ... Login And Run Command On Remote System
399 ... cat /tmp/rg_output | grep KBytes
400 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
401 Log ${output}
402 Should Contain ${output} KBytes
403
404 # Kill iperf on BNG
405 ${rg_output}= Run Keyword and Continue On Failure Login And Run Command On Remote System
406 ... sudo kill -9 `pidof iperf`
407 ... ${dst['bng_ip']} ${dst['bng_user']} ${dst['bng_pass']} ${dst['container_type']}
408 ... ${dst['container_name']}
409
410 # Kill iperf on the RG
411 ${output}= Run Keyword and Continue On Failure Login And Run Command On Remote System
412 ... sudo kill -9 `pidof iperf`
413 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
414
Suchitra Vemuri2f1e07d2020-06-26 19:18:01 -0700415
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000416Setup
417 [Documentation] Pre-test Setup
418 #test for empty device list
419 Test Empty Device List
Suchitra Vemuri3158f3c2020-05-28 17:58:26 -0700420 #Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s Openolt is Up
421 #... ${olt_ip} ${olt_user} ${olt_pass}
Suchitra Vemuri1d8e8172020-06-17 17:43:56 -0700422 Run Keyword If ${has_dataplane} Sleep 230s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000423 #create/preprovision device
424 ${olt_device_id}= Create Device ${olt_ip} ${OLT_PORT}
425 Set Suite Variable ${olt_device_id}
426 #validate olt states
427 Wait Until Keyword Succeeds ${timeout} 5s
ubuntu6b6e7d42020-03-02 12:35:42 -0800428 ... Validate OLT Device PREPROVISIONED UNKNOWN UNKNOWN ${olt_device_id}
Suchitra Vemuria7e8ad22020-02-11 18:02:40 -0800429 Sleep 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000430 Enable Device ${olt_device_id}
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700431 Wait Until Keyword Succeeds 380s 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000432 ... Validate OLT Device ENABLED ACTIVE REACHABLE ${olt_serial_number}
433 ${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
434 Set Suite Variable ${logical_id}
435
Hardik Windlassaaea3402020-03-10 19:45:45 +0530436Validate ONUs After OLT Disable
437 [Documentation] Validates the ONUs state in Voltha, ONUs port state in ONOS
438 ... and that pings do not succeed After corresponding OLT is Disabled
439 FOR ${I} IN RANGE 0 ${num_onus}
440 ${src}= Set Variable ${hosts.src[${I}]}
441 ${dst}= Set Variable ${hosts.dst[${I}]}
442 ${onu_device_id}= Get Device ID From SN ${src['onu']}
443 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
444 ... ${of_id}
445 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
446 ... Validate Device ENABLED DISCOVERED
447 ... UNREACHABLE ${src['onu']} onu=True onu_reason=stopping-openomci
448 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700449 ... Verify ONU Port Is Disabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Hardik Windlassaaea3402020-03-10 19:45:45 +0530450 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
451 ... Wait Until Keyword Succeeds 60s 2s
452 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
453 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
454 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
455 Run Keyword and Ignore Error Collect Logs
456 END
457
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000458Delete All Devices and Verify
459 [Documentation] Remove any devices from VOLTHA and ONOS
460 # Clear devices from VOLTHA
461 Disable Devices In Voltha Root=true
Suchitra Vemuri970747f2020-03-30 17:31:34 -0700462 Sleep 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000463 Wait Until Keyword Succeeds ${timeout} 2s Test Devices Disabled In Voltha Root=true
464 Delete Devices In Voltha Root=true
Matteo Scandolof6cea382020-04-20 14:32:00 -0700465 Run Keyword If ${has_dataplane} Sleep 30s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000466 Wait Until Keyword Succeeds ${timeout} 2s Test Empty Device List
467 # Clear devices from ONOS
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800468 #Remove All Devices From ONOS
ubuntu6b6e7d42020-03-02 12:35:42 -0800469 #... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000470
471Teardown
472 [Documentation] kills processes and cleans up interfaces on src+dst servers
473 Run Keyword If ${has_dataplane} Clean Up Linux
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000474
475Teardown Suite
476 [Documentation] Clean up device if desired
477 Run Keyword If ${teardown_device} Delete All Devices and Verify
478
479Delete Device and Verify
480 [Documentation] Disable -> Delete devices via voltctl and verify its removed
Suchitra Vemuri2fa9bba2020-01-22 17:38:48 -0800481 ${olt_device_id}= Get Device ID From SN ${olt_serial_number}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000482 ${rc} ${output}= Run and Return Rc and Output
483 ... ${VOLTCTL_CONFIG}; voltctl device disable ${olt_device_id}
484 Should Be Equal As Integers ${rc} 0
485 Sleep 5s
486 Wait Until Keyword Succeeds ${timeout} 5s
487 ... Validate OLT Device DISABLED UNKNOWN REACHABLE ${olt_serial_number}
488 ${rc} ${output}= Run and Return Rc and Output
489 ... ${VOLTCTL_CONFIG}; voltctl device delete ${olt_device_id}
Matteo Scandolof6cea382020-04-20 14:32:00 -0700490 Run Keyword If ${has_dataplane} Sleep 50s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000491 Should Be Equal As Integers ${rc} 0
492 Wait Until Keyword Succeeds ${timeout} 5s Validate Device Removed ${olt_device_id}
493
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000494Repeat Sanity Test
495 [Documentation] This keyword performs Sanity Test Procedure
496 ... Sanity test performs authentication, dhcp and pings for all the ONUs given for the POD
497 ... This keyword can be used to call in any other tests where sanity check is required
498 ... with wpa reassociation
499 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
500 Set Global Variable ${of_id}
501 FOR ${I} IN RANGE 0 ${num_onus}
Suchitra Vemuri03005912020-02-20 18:59:46 -0800502 Run Keyword and Ignore Error Collect Logs
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000503 ${src}= Set Variable ${hosts.src[${I}]}
504 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800505 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
506 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
507 # Check ONU port is Enabled in ONOD
508 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700509 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800510 # Verify EAPOL flows are added for the ONU port
511 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800512 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800513 # Verify ONU state in voltha
Suchitra Vemurid1e27fb2020-02-01 16:34:37 -0800514 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
Suchitra Vemuri81e4f022020-02-01 13:16:02 -0800515 ... ENABLED ACTIVE REACHABLE
516 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800517 # Perform Authentication
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000518 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
519 ... Validate Authentication After Reassociate
520 ... True ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']}
521 ... ${src['container_type']} ${src['container_name']}
522 Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800523 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000524 Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800525 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000526 ... volt-add-subscriber-access ${of_id} ${onu_port}
527 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
528 ... Validate DHCP and Ping True True
529 ... ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
530 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
531 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
532 ... ${dst['container_name']}
533 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800534 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000535 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
536 Run Keyword and Ignore Error Collect Logs
537 END
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000538
Hardik Windlass16cdf962020-04-29 15:26:50 +0530539Validate ONUs for PON OLT Disable
540 [Arguments] ${olt_peer_list}
541 [Documentation] This keyword validates that Ping fails for ONUs connected to Disabled OLT PON port
542 ... And Pings succeed for other Active OLT PON port ONUs
543 ... Also it removes subscriber for Disabled OLT PON port ONUs to replicate ATT workflow
544 FOR ${I} IN RANGE 0 ${num_onus}
545 ${src}= Set Variable ${hosts.src[${I}]}
546 ${dst}= Set Variable ${hosts.dst[${I}]}
547 ${onu_device_id}= Get Device ID From SN ${src['onu']}
548 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
549 ... ${of_id}
550 ${matched}= Match ONU in PON OLT Peer List ${olt_peer_list} ${onu_device_id}
551 Run Keyword If ${matched}
552 ... Run Keywords
553 ... Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
554 ... Validate Device ENABLED DISCOVERED
555 ... UNREACHABLE ${src['onu']} onu=True onu_reason=stopping-openomci
556 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
557 ... Verify ONU Port Is Disabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
558 ... AND Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
559 ... Wait Until Keyword Succeeds 60s 2s
560 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
561 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']}
562 ... ${src['container_name']}
563 # Remove Subscriber Access (To replicate ATT workflow)
564 ... AND Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${ONOS_SSH_IP}
565 ... ${ONOS_SSH_PORT} volt-remove-subscriber-access ${of_id} ${onu_port}
566 ... ELSE
567 ... Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
568 ... Wait Until Keyword Succeeds 60s 2s
569 ... Check Ping True ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
570 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']}
571 ... ${src['container_name']}
572 Run Keyword and Ignore Error Collect Logs
573 END
574
575Validate ONUs for PON OLT Enable
576 [Arguments] ${olt_peer_list}
577 [Documentation] This keyword validates Ping succeeds for all Enabled/Acitve OLT PON ports
578 ... Also performs Auth/subscriberAdd/DHCP/Ping for the ONUs on Re-Enabled OLT PON port
579 FOR ${I} IN RANGE 0 ${num_onus}
580 ${src}= Set Variable ${hosts.src[${I}]}
581 ${dst}= Set Variable ${hosts.dst[${I}]}
582 ${onu_device_id}= Get Device ID From SN ${src['onu']}
583 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
584 ... ${of_id}
585 ${matched}= Match ONU in PON OLT Peer List ${olt_peer_list} ${onu_device_id}
586 ${wpa_log}= Run Keyword If ${has_dataplane} and ${matched} Catenate SEPARATOR=.
587 ... /tmp/wpa ${src['dp_iface_name']} log
588 Run Keyword If ${matched}
589 ... Run Keywords
590 # Perform Cleanup
591 ... Run Keyword If ${has_dataplane} Clean Up Linux ${onu_device_id}
592 # Verify ONU port status
593 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
594 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
595 # Verify EAPOL flows are added for the ONU port
596 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
597 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
598 # Verify ONU state in voltha
599 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
600 ... Validate Device ENABLED ACTIVE REACHABLE
601 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
602 # Perform Authentication
603 ... AND Run Keyword If ${has_dataplane}
604 ... Run Keyword And Continue On Failure Validate Authentication True
605 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
606 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
607 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
608 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
609 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
610 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
611 ... volt-add-subscriber-access ${of_id} ${onu_port}
612 # Verify that no pending flows exist for the ONU port
613 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
614 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
615 # Verify subscriber access flows are added for the ONU port
616 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
617 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
618 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
619 ... AND Run Keyword If ${has_dataplane}
620 ... Run Keyword And Continue On Failure Validate DHCP and Ping True
621 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
622 ... ${src['ip']} ${src['user']} ${src['pass']}
623 ... ${src['container_type']} ${src['container_name']}
624 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']}
625 ... ${dst['container_type']} ${dst['container_name']}
626 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
627 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
628 ... AND Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
629 ... ELSE
630 ... Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
631 ... Wait Until Keyword Succeeds 60s 2s
632 ... Check Ping True ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
633 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']}
634 ... ${src['container_name']}
635 Run Keyword and Ignore Error Collect Logs
636 END
637
638Validate ONUs for PON OLT Disable DT
639 [Arguments] ${olt_peer_list}
640 [Documentation] This keyword validates that Ping fails for ONUs connected to Disabled OLT PON port
641 ... And Pings succeed for other Active OLT PON port ONUs
642 ... Also it removes subscriber and deletes ONUs for Disabled OLT PON port to replicate DT workflow
643 FOR ${I} IN RANGE 0 ${num_onus}
644 ${src}= Set Variable ${hosts.src[${I}]}
645 ${dst}= Set Variable ${hosts.dst[${I}]}
646 ${onu_device_id}= Get Device ID From SN ${src['onu']}
647 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
648 ... ${of_id}
649 ${matched}= Match ONU in PON OLT Peer List ${olt_peer_list} ${onu_device_id}
650 Run Keyword If ${matched}
651 ... Run Keywords
652 ... Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
653 ... Validate Device ENABLED DISCOVERED
654 ... UNREACHABLE ${src['onu']} onu=True onu_reason=stopping-openomci
655 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
656 ... Verify ONU Port Is Disabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
657 ... AND Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
658 ... Wait Until Keyword Succeeds 60s 2s
659 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
660 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']}
661 ... ${src['container_name']}
662 # Remove Subscriber Access (To replicate DT workflow)
663 ... AND Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${ONOS_SSH_IP}
664 ... ${ONOS_SSH_PORT} volt-remove-subscriber-access ${of_id} ${onu_port}
665 ... AND Sleep 10s
666 # Delete ONU Device (To replicate DT workflow)
667 ... AND Delete Device ${onu_device_id}
668 ... AND Sleep 5s
669 ... ELSE
670 ... Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
671 ... Wait Until Keyword Succeeds 60s 2s
672 ... Check Ping True ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
673 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']}
674 ... ${src['container_name']}
675 Run Keyword and Ignore Error Collect Logs
676 END
677
678Validate ONUs for PON OLT Enable DT
679 [Arguments] ${olt_peer_list}
680 [Documentation] This keyword validates Ping succeeds for all Enabled/Acitve OLT PON ports
681 ... Also performs subscriberAdd/DHCP/Ping for the ONUs on Re-Enabled OLT PON port
682 FOR ${I} IN RANGE 0 ${num_onus}
683 ${src}= Set Variable ${hosts.src[${I}]}
684 ${dst}= Set Variable ${hosts.dst[${I}]}
685 ${onu_device_id}= Get Device ID From SN ${src['onu']}
686 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
687 ... ${of_id}
688 ${matched}= Match ONU in PON OLT Peer List ${olt_peer_list} ${onu_device_id}
689 Run Keyword If ${matched}
690 ... Run Keywords
691 # Perform Cleanup
692 ... Run Keyword If ${has_dataplane} Clean Up Linux ${onu_device_id}
693 # Verify ONU port status
694 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
695 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
696 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
697 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
698 ... volt-add-subscriber-access ${of_id} ${onu_port}
699 # Verify ONU state in voltha
700 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
701 ... Validate Device ENABLED ACTIVE REACHABLE
702 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
703 # Verify subscriber access flows are added for the ONU port
704 ... AND Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
705 ... Verify Subscriber Access Flows Added For ONU DT ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
706 ... ${onu_port} ${nni_port} ${src['s_tag']}
707 ... AND Run Keyword If ${has_dataplane}
708 ... Run Keyword And Continue On Failure Validate DHCP and Ping True
709 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
710 ... ${src['ip']} ${src['user']} ${src['pass']}
711 ... ${src['container_type']} ${src['container_name']}
712 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']}
713 ... ${dst['container_type']} ${dst['container_name']}
714 ... ELSE
715 ... Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
716 ... Wait Until Keyword Succeeds 60s 2s
717 ... Check Ping True ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
718 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']}
719 ... ${src['container_name']}
720 Run Keyword and Ignore Error Collect Logs
721 END
722
723Match ONU in PON OLT Peer List
724 [Arguments] ${olt_peer_list} ${onu_device_id}
725 [Documentation] This keyword matches if ONU device is present in OLT PON port peer list
726 ${matched}= Set Variable False
727 FOR ${olt_peer} IN @{olt_peer_list}
728 ${matched}= Set Variable If '${onu_device_id}' == '${olt_peer}' True False
729 Exit For Loop If ${matched}
730 END
731 [Return] ${matched}
732
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000733Collect Logs
734 [Documentation] Collect Logs from voltha and onos cli for various commands
735 Run Keyword and Ignore Error Get Device List from Voltha
736 Run Keyword and Ignore Error Get Device Output from Voltha ${olt_device_id}
737 Run Keyword and Ignore Error Get Logical Device Output from Voltha ${logical_id}
ubuntu6b6e7d42020-03-02 12:35:42 -0800738 Get ONOS Status ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvanf68ea4b2020-02-03 07:36:39 +0000739
740Verify ping is succesful except for given device
741 [Arguments] ${num_onus} ${exceptional_onu_id}
742 [Documentation] Checks that ping for all the devices are successful except the given ONU.
743 ${pingStatus} Set Variable True
744 FOR ${I} IN RANGE 0 ${num_onus}
745 ${src}= Set Variable ${hosts.src[${I}]}
746 ${dst}= Set Variable ${hosts.dst[${I}]}
747 ${onu_device_id}= Get Device ID From SN ${src['onu']}
748 ${pingStatus} Run Keyword If '${onu_device_id}' == '${exceptional_onu_id}' Set Variable False
749 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 60s 2s
750 ... Check Ping ${pingStatus} ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
751 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
752 END
Andy Bavier4a8450e2020-02-04 08:58:37 -0700753
Andy Bavier1923ecc2020-06-18 14:06:40 -0700754Echo Message to OLT Logs
755 [Arguments] ${message}
756 [Documentation] Echoes ${message} into the OLT logs
Andy Bavier690ce432020-06-20 16:31:10 -0700757 Wait Until Keyword Succeeds 180s 10s Execute Remote Command
Andy Bavier6f613612020-06-19 15:08:30 -0700758 ... printf '%s\n' '' '' '${message}' '' >> /var/log/openolt.log
Suchitra Vemurib7253a52020-07-14 22:35:17 -0700759 ... ${olt_ssh_ip} ${olt_user} ${olt_pass}
Andy Bavier690ce432020-06-20 16:31:10 -0700760 Wait Until Keyword Succeeds 180s 10s Execute Remote Command
Andy Bavier6f613612020-06-19 15:08:30 -0700761 ... printf '%s\n' '' '' '${message}' '' >> /var/log/dev_mgmt_daemon.log
Suchitra Vemurib7253a52020-07-14 22:35:17 -0700762 ... ${olt_ssh_ip} ${olt_user} ${olt_pass}
Girish Gowdra26711d52020-07-28 15:57:02 -0700763 Wait Until Keyword Succeeds 180s 10s Execute Remote Command
764 ... printf '%s\n' '' '' '${message}' '' >> /var/log/openolt_process_watchdog.log
765 ... ${olt_ssh_ip} ${olt_user} ${olt_pass}
Andy Bavier1923ecc2020-06-18 14:06:40 -0700766
Andy Bavierabeba262020-02-07 16:22:16 -0700767Start Logging
768 [Arguments] ${label}
769 [Documentation] Start logging for test ${label}
770 ${kail_process}= Run Keyword If "${container_log_dir}" != "${None}" Start Process kail -n default
771 ... -n voltha cwd=${container_log_dir} stdout=${label}-combined.log
772 Set Test Variable ${kail_process}
Andy Bavier1923ecc2020-06-18 14:06:40 -0700773 Run Keyword If ${has_dataplane} Echo Message to OLT Logs START ${label}
Andy Bavierabeba262020-02-07 16:22:16 -0700774
775Stop Logging
776 [Arguments] ${label}
777 [Documentation] End logging for test; remove logfile if test passed
778 Run sync
779 Run Keyword If ${kail_process} Terminate Process ${kail_process}
780 ${test_logfile}= Run Keyword If "${container_log_dir}" != "${None}"
781 ... Join Path ${container_log_dir} ${label}-combined.log
782 Run Keyword If Test Passed Run Keyword If "${test_logfile}" != "${None}" Remove File ${test_logfile}
Andy Bavier1923ecc2020-06-18 14:06:40 -0700783 Run Keyword If ${has_dataplane} Echo Message to OLT Logs END ${label}
Andy Bavierabeba262020-02-07 16:22:16 -0700784
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800785Clean Up Linux
786 [Documentation] Kill processes and clean up interfaces on src+dst servers
Hardik Windlass16cdf962020-04-29 15:26:50 +0530787 [Arguments] ${onu_id}=${EMPTY}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800788 FOR ${I} IN RANGE 0 ${num_onus}
789 ${src}= Set Variable ${hosts.src[${I}]}
790 ${dst}= Set Variable ${hosts.dst[${I}]}
Hardik Windlass16cdf962020-04-29 15:26:50 +0530791 ${onu_device_id}= Get Device ID From SN ${src['onu']}
792 Continue For Loop If '${onu_id}' != '${EMPTY}' and '${onu_id}' != '${onu_device_id}'
Andy Bavier84b83282020-06-30 05:47:49 -0700793 Execute Remote Command sudo pkill wpa_supplicant ${src['ip']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800794 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Andy Bavier84b83282020-06-30 05:47:49 -0700795 Execute Remote Command sudo pkill dhclient ${src['ip']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800796 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Andy Baviereff938d2020-06-29 18:27:49 -0700797 Execute Remote Command sudo pkill mausezahn ${src['ip']}
798 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
799 Run Keyword If '${dst['ip']}' != '${None}' Execute Remote Command pkill dhcpd
800 ... ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']} ${dst['container_name']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800801 Delete IP Addresses from Interface on Remote Host ${src['dp_iface_name']} ${src['ip']}
802 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
803 Run Keyword If '${dst['ip']}' != '${None}' Delete Interface on Remote Host
804 ... ${dst['dp_iface_name']}.${src['s_tag']} ${dst['ip']} ${dst['user']} ${dst['pass']}
805 ... ${dst['container_type']} ${dst['container_name']}
Andy Bavier84b83282020-06-30 05:47:49 -0700806 ${bng_ip}= Get Variable Value ${dst['noroot_ip']}
807 ${bng_user}= Get Variable Value ${dst['noroot_user']}
808 ${bng_pass}= Get Variable Value ${dst['noroot_pass']}
Andy Bavierf4b7d1d2020-06-30 11:14:04 -0700809 Run Keyword If "${bng_ip}" != "${NONE}" and "${bng_user}" != "${NONE}" and "${bng_pass}" != "${NONE}"
810 ... Execute Remote Command sudo pkill mausezahn ${bng_ip} ${bng_user} ${bng_pass}
Andy Baviereff938d2020-06-29 18:27:49 -0700811 ... ${dst['container_type']} ${dst['container_name']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800812 END
Scott Baker60e570d2020-02-02 22:10:13 -0800813
Suchitra Vemuri8d3b0032020-09-11 17:49:08 -0700814Clean dhclient
815 [Documentation] Kills dhclient processes only for all RGs
816 FOR ${I} IN RANGE 0 ${num_onus}
817 ${src}= Set Variable ${hosts.src[${I}]}
818 ${dst}= Set Variable ${hosts.dst[${I}]}
819 Execute Remote Command sudo pkill dhclient ${src['ip']}
820 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
821 END
822
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800823Clean WPA Process
824 [Documentation] Kills wpa_supplicant processes only for all RGs
825 FOR ${I} IN RANGE 0 ${num_onus}
826 ${src}= Set Variable ${hosts.src[${I}]}
827 ${dst}= Set Variable ${hosts.dst[${I}]}
828 Run Keyword And Ignore Error Kill Linux Process [w]pa_supplicant ${src['ip']}
829 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
830 END
831
Scott Baker60e570d2020-02-02 22:10:13 -0800832Should Be Larger Than
833 [Documentation] Verify that value_1 is > value_2
834 [Arguments] ${value_1} ${value_2}
835 Run Keyword If ${value_1} <= ${value_2}
836 ... Fail The value ${value_1} is not larger than ${value_2}
837
Scott Bakeree675552020-02-11 10:43:34 -0800838Should Be Larger Than Or Equal To
839 [Documentation] Verify that value_1 is >= value_2
840 [Arguments] ${value_1} ${value_2}
841 Run Keyword If ${value_1} < ${value_2}
842 ... Fail The value ${value_1} is not larger than or equal to ${value_2}
843
Scott Baker60e570d2020-02-02 22:10:13 -0800844Should Be Float
845 [Documentation] Verify that value is a floating point number type
846 [Arguments] ${value}
847 ${type} Evaluate type(${value}).__name__
848 Should Be Equal ${type} float
849
Scott Bakeree675552020-02-11 10:43:34 -0800850Should Be Newer Than Or Equal To
851 [Documentation] Compare two RFC3339 dates
852 [Arguments] ${value_1} ${value_2}
853 ${unix_v1} Parse RFC3339 ${value_1}
854 ${unix_v2} Parse RFC3339 ${value_2}
855 Run Keyword If ${unix_v1} < ${unix_v2}
856 ... Fail The value ${value_1} is not newer than or equal to ${value_2}
857
Scott Baker60e570d2020-02-02 22:10:13 -0800858Get Current Time
859 [Documentation] Return the current time in RFC3339 format
860 ${output}= Run date -u +"%FT%T%:z"
861 [return] ${output}
Scott Bakeree675552020-02-11 10:43:34 -0800862
863Parse RFC3339
864 [Documentation] Parse an RFC3339 timestamp
865 [Arguments] ${dateStr}
866 ${rc} ${output}= Run and Return Rc and Output date --date="${dateStr}" "+%s"
867 Should Be Equal As Numbers ${rc} 0
868 [return] ${output}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700869
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000870Get Bandwidth Profile Name For Given Subscriber
871 [Arguments] ${subscriber_id} ${stream_type}=upstreamBandwidthProfile
872 [Documentation] Keyword to get the bandwidth details of the given subscriber
873 ${bandwidth_profile_output}= Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
874 ... volt-programmed-subscribers | grep ${subscriber_id}
875 @{bandwidth_profile_array}= Split String ${bandwidth_profile_output} ,
876 Log ${bandwidth_profile_array}
877 FOR ${value} IN @{bandwidth_profile_array}
878 @{row_value}= Split String ${value} =
879 ${bandwidth_profile_name}= Set Variable If '${row_value[0]}' == ' ${stream_type}'
880 ... ${row_value[1]}
881 ${bandwidth_profile_name}= Convert To String ${bandwidth_profile_name}
882 Run Keyword If "${bandwidth_profile_name}" != "None" Exit For Loop
883 END
884 Log ${bandwidth_profile_name}
885 [Return] ${bandwidth_profile_name}
886
887Execute Remote Command
888 [Documentation] SSH into a remote host and execute a command on the bare host or in a container.
889 ... This replaces and simplifies the Login And Run Command On Remote System keyword in CORDRobot.
890 [Arguments] ${cmd} ${ip} ${user} ${pass}=${None}
Andy Baviereff938d2020-06-29 18:27:49 -0700891 ... ${container_type}=${None} ${container_name}=${None} ${timeout}=${None}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000892 ${conn_id}= SSHLibrary.Open Connection ${ip}
893 Run Keyword If '${pass}' != '${None}'
894 ... SSHLibrary.Login ${user} ${pass}
895 ... ELSE
896 ... SSHLibrary.Login With Public Key ${user} %{HOME}/.ssh/id_rsa
897 ${namespace}= Run Keyword If '${container_type}' == 'K8S' SSHLibrary.Execute Command
898 ... kubectl get pods --all-namespaces | grep ${container_name} | awk '{print $1}'
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700899 ${stdout} ${stderr} ${rc}= Run Keyword If '${container_type}' == 'LXC'
900 ... SSHLibrary.Execute Command lxc exec ${container_name} -- ${cmd}
Andy Baviereff938d2020-06-29 18:27:49 -0700901 ... return_stderr=True return_rc=True timeout=${timeout}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000902 ... ELSE IF '${container_type}' == 'K8S'
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700903 ... SSHLibrary.Execute Command kubectl -n ${namespace} exec ${container_name} -- ${cmd}
Andy Baviereff938d2020-06-29 18:27:49 -0700904 ... return_stderr=True return_rc=True timeout=${timeout}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000905 ... ELSE
Andy Baviereff938d2020-06-29 18:27:49 -0700906 ... SSHLibrary.Execute Command ${cmd} return_stderr=True return_rc=True timeout=${timeout}
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700907
908 Log ${stdout}
909 Log ${stderr}
910 Log ${rc}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000911 SSHLibrary.Close Connection
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700912 [Return] ${stdout} ${stderr} ${rc}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000913
Suraj Gour33651272020-05-04 10:10:16 +0530914Start Remote Command
915 [Documentation] SSH into a remote host and execute a command on the bare host or in a container.
916 ... This replaces and simplifies the Login And Run Command On Remote System keyword in CORDRobot.
917 [Arguments] ${cmd} ${ip} ${user} ${pass}=${None}
918 ... ${container_type}=${None} ${container_name}=${None}
919 ${conn_id}= SSHLibrary.Open Connection ${ip}
920 Run Keyword If '${pass}' != '${None}'
921 ... SSHLibrary.Login ${user} ${pass}
922 ... ELSE
923 ... SSHLibrary.Login With Public Key ${user} %{HOME}/.ssh/id_rsa
924 ${namespace}= Run Keyword If '${container_type}' == 'K8S' SSHLibrary.Execute Command
925 ... kubectl get pods --all-namespaces | grep ${container_name} | awk '{print $1}'
926 Run Keyword If '${container_type}' == 'LXC'
927 ... SSHLibrary.Start Command lxc exec ${container_name} -- ${cmd}
928 ... ELSE IF '${container_type}' == 'K8S'
929 ... SSHLibrary.Start Command kubectl -n ${namespace} exec ${container_name} -- ${cmd}
930 ... ELSE
931 ... SSHLibrary.Start Command ${cmd}
Andy Bavier9fb23232020-06-30 15:12:20 -0700932 # It seems that closing the connection immediately will sometimes kill the command
933 Sleep 1s
Suraj Gour33651272020-05-04 10:10:16 +0530934 SSHLibrary.Close Connection
935
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000936Run Iperf3 Test Client
937 [Arguments] ${src} ${server} ${args}
938 [Documentation] Login to ${src} and run the iperf3 client against ${server} using ${args}.
939 ... Return a Dictionary containing the results of the test.
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700940 ${output} ${stderr} ${rc}= Execute Remote Command iperf3 -J -c ${server} ${args} | jq -M -c '.'
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000941 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700942 Should Be Equal As Integers ${rc} 0
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000943 ${object}= Evaluate json.loads(r'''${output}''') json
944 [Return] ${object}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000945
Suchitra Vemuri85cb7d62020-07-16 23:46:16 -0700946Run Iperf Test Client for MCAST
947 [Arguments] ${src} ${server} ${args}
948 [Documentation] Login to ${src} and run the iperf client against ${server} using ${args}.
949 ... Return a Dictionary containing the results of the test.
950 ${output} ${stderr} ${rc}= Execute Remote Command sudo iperf -c ${server} ${args} | jq -M -c '.'
951 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
952 Should Be Equal As Integers ${rc} 0
953 ${object}= Evaluate json.loads(r'''${output}''') json
954 [Return] ${object}
955
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000956RestoreONUs
957 [Documentation] Restore all connected ONUs
958 [Arguments] ${num_onus}
959 FOR ${I} IN RANGE 0 ${num_onus}
960 ${src}= Set Variable ${hosts.src[${I}]}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700961 ${container_type}= Get Variable Value ${src['container_type']} "null"
962 ${container_name}= Get Variable Value ${src['container_name']} "null"
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000963 ${onu_type}= Get Variable Value ${src['onu_type']} "null"
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700964 #Get ens6f0 from ens6f0.22
965 ${if_name}= Replace String Using Regexp ${src['dp_iface_name']} \\..* \
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000966 Run Keyword IF '${onu_type}' == 'alpha' AlphaONURestoreDefault ${src['ip']} ${src['user']}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700967 ... ${src['pass']} ${if_name} admin admin ${container_type} ${container_name}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000968 END
969
970AlphaONURestoreDefault
971 [Documentation] Restore the Alpha ONU to factory setting
972 [Arguments] ${rg_ip} ${rg_user} ${rg_pass} ${onu_ifname}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700973 ... ${onu_user} ${onu_pass} ${container_type}=${None} ${container_name}=${None}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000974 ${output}= Login And Run Command On Remote System sudo ifconfig ${onu_ifname} 192.168.1.3/24
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700975 ... ${rg_ip} ${rg_user} ${rg_pass} ${container_type} ${container_name}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000976 ${cmd} Catenate
977 ... (echo open "192.168.1.1"; sleep 1;
978 ... echo "${onu_user}"; sleep 1;
979 ... echo "${onu_pass}"; sleep 1;
980 ... echo "restoredefault"; sleep 1) | telnet
981 ${output}= Login And Run Command On Remote System ${cmd}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700982 ... ${rg_ip} ${rg_user} ${rg_pass} ${container_type} ${container_name}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000983 Log To Console ${output}
984 ${output}= Login And Run Command On Remote System sudo ifconfig ${onu_ifname} 0
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700985 ... ${rg_ip} ${rg_user} ${rg_pass} ${container_type} ${container_name}
Suraj Gour33651272020-05-04 10:10:16 +0530986
987Create traffic with each pbit and capture at other end
988 [Documentation] Generates upstream traffic using Mausezahn tool
989 ... with each pbit and validates reception at other end using tcpdump
990 [Arguments] ${target_ip} ${target_iface} ${src_iface}
991 ... ${packet_count} ${packet_type} ${target_port} ${vlan} ${tcpdump_filter}
992 ... ${dst_ip} ${dst_user} ${dst_pass} ${dst_container_type} ${dst_container_name}
993 ... ${src_ip} ${src_user} ${src_pass} ${src_container_type} ${src_container_name}
994 FOR ${pbit} IN RANGE 8
Andy Baviereff938d2020-06-29 18:27:49 -0700995 Execute Remote Command sudo pkill mausezahn
Suraj Gour33651272020-05-04 10:10:16 +0530996 ... ${src_ip} ${src_user} ${src_pass} ${src_container_type} ${src_container_name}
Andy Baviereff938d2020-06-29 18:27:49 -0700997 ${var1}= Set Variable sudo mausezahn ${src_iface} -B ${target_ip} -c ${packet_count} -d 100m
Suraj Gour33651272020-05-04 10:10:16 +0530998 ${var2}= Set Variable -t ${packet_type} "dp=${target_port}" -p 1472 -Q ${pbit}:${vlan}
999 ${cmd}= Set Variable ${var1} ${var2}
1000 Start Remote Command ${cmd} ${src_ip} ${src_user} ${src_pass}
1001 ... ${src_container_type} ${src_container_name}
1002 ${output} ${stderr} ${rc}= Execute Remote Command
Andy Baviereff938d2020-06-29 18:27:49 -07001003 ... sudo tcpdump -l -U -c 30 -i ${target_iface} -e ${tcpdump_filter}
Suraj Gour33651272020-05-04 10:10:16 +05301004 ... ${dst_ip} ${dst_user} ${dst_pass} ${dst_container_type} ${dst_container_name}
Andy Baviereff938d2020-06-29 18:27:49 -07001005 ... timeout=30 seconds
1006 Execute Remote Command sudo pkill mausezahn
Suraj Gour33651272020-05-04 10:10:16 +05301007 ... ${src_ip} ${src_user} ${src_pass} ${src_container_type} ${src_container_name}
1008 # VOL-3262: I'm seeing untagged downstream traffic at RG for pbit 0. According to Girish this is
1009 # incorrect behavior. Simplify the following check when VOL-3262 is resolved.
1010 Run Keyword If ${pbit}==0 and "${tcpdump_filter}"=="udp"
1011 ... Should Match Regexp ${output} \\.${target_port}: UDP,
1012 ... ELSE Should Match Regexp ${output} , p ${pbit},.*\\.${target_port}: UDP,
1013 END