[VOL-2473]Sanity Test keyword
Change-Id: I3a9e9e79784eccb9519c89c82e7f64a12ee49532
diff --git a/libraries/utils.robot b/libraries/utils.robot
index a4ad0f4..1ecde32 100644
--- a/libraries/utils.robot
+++ b/libraries/utils.robot
@@ -142,3 +142,96 @@
... cat ${file} | grep '${pattern}' | wc -l ${ip} ${user} ${pass}
... ${container_type} ${container_name} ${prompt}
[Return] ${result}
+
+Perform Sanity Test
+ [Documentation] This keyword performs Sanity Test Procedure
+ ... Sanity test performs authentication, dhcp and pings for all the ONUs given for the POD
+ ... This keyword can be used to call in any other tests where sanity check is required
+ ... and avoids duplication of code.
+ ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
+ Set Global Variable ${of_id}
+ FOR ${I} IN RANGE 0 ${num_onus}
+ ${src}= Set Variable ${hosts.src[${I}]}
+ ${dst}= Set Variable ${hosts.dst[${I}]}
+ Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
+ ... ENABLED ACTIVE REACHABLE
+ ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
+ ${onu_device_id}= Get Device ID From SN ${src['onu']}
+ ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
+ ... Get ONU Port in ONOS ${src['onu']} ${of_id}
+ Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
+ ... Verify Eapol Flows Added For ONU ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
+ Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
+ ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
+ ... ${src['container_type']} ${src['container_name']}
+ Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
+ ... Verify ONU in AAA-Users ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
+ Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
+ ... Execute ONOS CLI Command ${k8s_node_ip} ${ONOS_SSH_PORT}
+ ... volt-add-subscriber-access ${of_id} ${onu_port}
+ Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
+ ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
+ ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
+ ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
+ ... ${dst['container_name']}
+ Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
+ ... Validate Subscriber DHCP Allocation ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
+ Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
+ Run Keyword and Ignore Error Collect Logs
+ END
+
+Setup
+ [Documentation] Pre-test Setup
+ #test for empty device list
+ Test Empty Device List
+ Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s Openolt is Up
+ ... ${olt_ip} ${olt_user} ${olt_pass}
+ Sleep 60s
+ #create/preprovision device
+ ${olt_device_id}= Create Device ${olt_ip} ${OLT_PORT}
+ Set Suite Variable ${olt_device_id}
+ #validate olt states
+ Wait Until Keyword Succeeds ${timeout} 5s
+ ... Validate OLT Device PREPROVISIONED UNKNOWN UNKNOWN ${EMPTY} ${olt_device_id}
+ Enable Device ${olt_device_id}
+ Wait Until Keyword Succeeds ${timeout} 5s
+ ... Validate OLT Device ENABLED ACTIVE REACHABLE ${olt_serial_number}
+ ${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
+ Set Suite Variable ${logical_id}
+
+Delete All Devices and Verify
+ [Documentation] Remove any devices from VOLTHA and ONOS
+ # Clear devices from VOLTHA
+ Disable Devices In Voltha Root=true
+ Wait Until Keyword Succeeds ${timeout} 2s Test Devices Disabled In Voltha Root=true
+ Delete Devices In Voltha Root=true
+ Wait Until Keyword Succeeds ${timeout} 2s Test Empty Device List
+ # Clear devices from ONOS
+ Remove All Devices From ONOS
+ ... http://karaf:karaf@${k8s_node_ip}:${ONOS_REST_PORT}
+
+Teardown
+ [Documentation] kills processes and cleans up interfaces on src+dst servers
+ Run Keyword If ${has_dataplane} Clean Up Linux
+ Run Keyword If ${external_libs} Run Keyword and Ignore Error
+ ... Log Kubernetes Containers Logs Since Time
+ ... ${datetime} ${container_list}
+
+Teardown Suite
+ [Documentation] Clean up device if desired
+ Run Keyword If ${teardown_device} Delete All Devices and Verify
+
+Delete Device and Verify
+ [Documentation] Disable -> Delete devices via voltctl and verify its removed
+ ${rc} ${output}= Run and Return Rc and Output
+ ... ${VOLTCTL_CONFIG}; voltctl device disable ${olt_device_id}
+ Should Be Equal As Integers ${rc} 0
+ Sleep 5s
+ Wait Until Keyword Succeeds ${timeout} 5s
+ ... Validate OLT Device DISABLED UNKNOWN REACHABLE ${olt_serial_number}
+ ${rc} ${output}= Run and Return Rc and Output
+ ... ${VOLTCTL_CONFIG}; voltctl device delete ${olt_device_id}
+ Sleep 50s
+ Should Be Equal As Integers ${rc} 0
+ Wait Until Keyword Succeeds ${timeout} 5s Validate Device Removed ${olt_device_id}
+
diff --git a/tests/functional/Voltha_ErrorScenarios.robot b/tests/functional/Voltha_ErrorScenarios.robot
new file mode 100644
index 0000000..76908b5
--- /dev/null
+++ b/tests/functional/Voltha_ErrorScenarios.robot
@@ -0,0 +1,78 @@
+# Copyright 2017 - present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# FIXME Can we use the same test against BBSim and Hardware?
+
+*** Settings ***
+Documentation Test various end-to-end scenarios
+Suite Setup Common Test Suite Setup
+Test Setup Setup
+Test Teardown Teardown
+#Suite Teardown Teardown Suite
+Library Collections
+Library String
+Library OperatingSystem
+Library XML
+Library RequestsLibrary
+Library ../../libraries/DependencyLibrary.py
+Resource ../../libraries/onos.robot
+Resource ../../libraries/voltctl.robot
+Resource ../../libraries/voltha.robot
+Resource ../../libraries/utils.robot
+Resource ../../libraries/k8s.robot
+Resource ../../variables/variables.robot
+
+*** Variables ***
+${POD_NAME} flex-ocp-cord
+${KUBERNETES_CONF} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
+${KUBERNETES_CONFIGS_DIR} ~/pod-configs/kubernetes-configs
+#${KUBERNETES_CONFIGS_DIR} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
+${KUBERNETES_YAML} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.yml
+${HELM_CHARTS_DIR} ~/helm-charts
+${VOLTHA_POD_NUM} 8
+${NAMESPACE} voltha
+# For below variable value, using deployment name as using grep for
+# parsing radius pod name, we can also use full radius pod name
+${RESTART_POD_NAME} radius
+${timeout} 60s
+${of_id} 0
+${logical_id} 0
+${has_dataplane} False
+${external_libs} True
+${teardown_device} False
+${scripts} ../../scripts
+
+*** Test Cases ***
+Adding the same OLT before and after enabling the device
+ [Documentation] Create OLT, Create the same OLT again and Check for the Error message
+ [Tags] VOL-2405 VOL-2406 AddSameOLT notready
+ [Setup] None
+ [Teardown] None
+ ${olt_device_id}= Create Device ${olt_ip} ${OLT_PORT}
+ Set Suite Variable ${olt_device_id}
+ Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Device PREPROVISIONED UNKNOWN UNKNOWN
+ ... ${EMPTY} ${olt_device_id}
+ ${rc} ${output}= Run and Return Rc and Output
+ ... ${VOLTCTL_CONFIG}; voltctl device create -t openolt -H ${olt_ip}:${OLT_PORT}
+ Should Not Be Equal As Integers ${rc} 0
+ Should Contain ${output} Device is already pre-provisioned
+ #Enable the created OLT device
+ Enable Device ${olt_device_id}
+ Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Device ENABLED ACTIVE REACHABLE
+ ... ${olt_serial_number}
+ ${rc} ${output}= Run and Return Rc and Output
+ ... ${VOLTCTL_CONFIG}; voltctl device create -t openolt -H ${olt_ip}:${OLT_PORT}
+ Should Not Be Equal As Integers ${rc} 0
+ Log ${output}
+ Should Contain ${output} Device is already pre-provisioned
+ Log "This OLT is added already and enabled"
diff --git a/tests/functional/Voltha_PODTests.robot b/tests/functional/Voltha_PODTests.robot
index 99f2938..1754646 100644
--- a/tests/functional/Voltha_PODTests.robot
+++ b/tests/functional/Voltha_PODTests.robot
@@ -331,7 +331,7 @@
... Disable and Delete the OLT
... Create/Enable the same OLT again
... Validate authentication/DHCP/E2E pings succeed for all the ONUs connected to the OLT
- [Tags] functional DeleteOLT
+ [Tags] functional DeleteOLT notready
[Setup] None
[Teardown] NONE
Run Keyword If ${has_dataplane} Clean Up Linux
@@ -342,21 +342,6 @@
Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
Run Keyword and Ignore Error Collect Logs
-Adding the same OLT before enabling the device
- [Documentation] Create OLT, Create the same OLT again and Check for the Error message
- [Tags] VOL-2405 AddSameOLT notready
- [Setup] Delete Device and Verify
- [Teardown] Enable Device ${olt_device_id}
- Run Keyword If ${has_dataplane} Sleep 180s
- ${olt_device_id}= Create Device ${olt_ip} ${OLT_PORT}
- Set Suite Variable ${olt_device_id}
- Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Device PREPROVISIONED UNKNOWN UNKNOWN
- ... ${EMPTY} ${olt_device_id}
- ${rc} ${output}= Run and Return Rc and Output
- ... ${VOLTCTL_CONFIG}; voltctl device create -t openolt -H ${olt_ip}:${OLT_PORT}
- Should Not Be Equal As Integers ${rc} 0
- Should Contain ${output} Device is already pre-provisioned
-
Adding the same OLT after enabling the device
[Documentation] Create OLT, enable it, Create the same OLT again and Check for the Error message
[Tags] VOL-2406 AddEnableOLT_AddTheSameOLTAgain functional
@@ -440,36 +425,6 @@
[Documentation] Set up the test suite
Common Test Suite Setup
-Setup
- [Documentation] Pre-test Setup
- #test for empty device list
- Test Empty Device List
- Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s Openolt is Up
- ... ${olt_ip} ${olt_user} ${olt_pass}
- Sleep 60s
- #create/preprovision device
- ${olt_device_id}= Create Device ${olt_ip} ${OLT_PORT}
- Set Suite Variable ${olt_device_id}
- #validate olt states
- Wait Until Keyword Succeeds ${timeout} 5s
- ... Validate OLT Device PREPROVISIONED UNKNOWN UNKNOWN ${EMPTY} ${olt_device_id}
- Enable Device ${olt_device_id}
- Wait Until Keyword Succeeds ${timeout} 5s
- ... Validate OLT Device ENABLED ACTIVE REACHABLE ${olt_serial_number}
- ${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
- Set Suite Variable ${logical_id}
-
-Delete All Devices and Verify
- [Documentation] Remove any devices from VOLTHA and ONOS
- # Clear devices from VOLTHA
- Disable Devices In Voltha Root=true
- Wait Until Keyword Succeeds ${timeout} 2s Test Devices Disabled In Voltha Root=true
- Delete Devices In Voltha Root=true
- Wait Until Keyword Succeeds ${timeout} 2s Test Empty Device List
- # Clear devices from ONOS
- Remove All Devices From ONOS
- ... http://karaf:karaf@${k8s_node_ip}:${ONOS_REST_PORT}
-
Clear All Devices Then Create New Device
[Documentation] Remove any devices from VOLTHA and ONOS
# Remove all devices from voltha and nos
@@ -477,13 +432,6 @@
# Execute normal test Setup Keyword
Setup
-Teardown
- [Documentation] kills processes and cleans up interfaces on src+dst servers
- Run Keyword If ${has_dataplane} Clean Up Linux
- Run Keyword If ${external_libs} Run Keyword and Ignore Error
- ... Log Kubernetes Containers Logs Since Time
- ... ${datetime} ${container_list}
-
Collect Logs
[Documentation] Collect Logs from voltha and onos cli for various commands
Run Keyword and Ignore Error Get Device List from Voltha
@@ -491,10 +439,6 @@
Run Keyword and Ignore Error Get Logical Device Output from Voltha ${logical_id}
Run Keyword If ${external_libs} Get ONOS Status ${k8s_node_ip}
-Teardown Suite
- [Documentation] Clean up device if desired
- Run Keyword If ${teardown_device} Delete All Devices and Verify
-
Clean Up Linux
[Documentation] Kill processes and clean up interfaces on src+dst servers
FOR ${I} IN RANGE 0 ${num_onus}
@@ -514,57 +458,6 @@
... ${dst['container_type']} ${dst['container_name']}
END
-Delete Device and Verify
- [Documentation] Disable -> Delete devices via voltctl and verify its removed
- ${rc} ${output}= Run and Return Rc and Output
- ... ${VOLTCTL_CONFIG}; voltctl device disable ${olt_device_id}
- Should Be Equal As Integers ${rc} 0
- Sleep 5s
- Wait Until Keyword Succeeds ${timeout} 5s
- ... Validate OLT Device DISABLED UNKNOWN REACHABLE ${olt_serial_number}
- ${rc} ${output}= Run and Return Rc and Output
- ... ${VOLTCTL_CONFIG}; voltctl device delete ${olt_device_id}
- Sleep 50s
- Should Be Equal As Integers ${rc} 0
- Wait Until Keyword Succeeds ${timeout} 5s Validate Device Removed ${olt_device_id}
-
-Perform Sanity Test
- [Documentation] This keyword performs Sanity Test Procedure
- ... Sanity test performs authentication, dhcp and pings for all the ONUs given for the POD
- ... This keyword can be used to call in any other tests where sanity check is required
- ... and avoids duplication of code.
- ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
- Set Global Variable ${of_id}
- FOR ${I} IN RANGE 0 ${num_onus}
- ${src}= Set Variable ${hosts.src[${I}]}
- ${dst}= Set Variable ${hosts.dst[${I}]}
- Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
- ... ENABLED ACTIVE REACHABLE
- ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
- ${onu_device_id}= Get Device ID From SN ${src['onu']}
- ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
- ... Get ONU Port in ONOS ${src['onu']} ${of_id}
- Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
- ... Verify Eapol Flows Added For ONU ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
- Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
- ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
- ... ${src['container_type']} ${src['container_name']}
- Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
- ... Verify ONU in AAA-Users ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
- Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
- ... Execute ONOS CLI Command ${k8s_node_ip} ${ONOS_SSH_PORT}
- ... volt-add-subscriber-access ${of_id} ${onu_port}
- Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
- ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
- ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
- ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
- ... ${dst['container_name']}
- Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
- ... Validate Subscriber DHCP Allocation ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
- Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
- Run Keyword and Ignore Error Collect Logs
- END
-
Repeat Sanity Test
[Documentation] Performs Sanity Test Procedure where sanity check is required with wpa reassociation
${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}