[VOL-2885] updations to tests
Change-Id: I514f47817624500b95b36acf5aac87a4cf0560fa
(cherry picked from commit ebf6532b2c872e31890b1788bf2ab67400946903)
diff --git a/tests/functional/Voltha_FailureScenarios.robot b/tests/functional/Voltha_FailureScenarios.robot
index 564f381..904919a 100644
--- a/tests/functional/Voltha_FailureScenarios.robot
+++ b/tests/functional/Voltha_FailureScenarios.robot
@@ -219,77 +219,6 @@
Run Keyword and Ignore Error Collect Logs
END
-Sanity E2E Test for OLT/ONU on POD With Core Fail and Restart
- [Documentation] Deploys an device instance and waits for it to authenticate. After
- ... authentication is successful the rw-core deployment is scaled to 0 instances to
- ... simulate a POD crash. The test then scales the rw-core back to a single instance
- ... and configures ONOS for access. The test succeeds if the device is able to
- ... complete the DHCP sequence.
- [Tags] functional rwcore-restart notready
- [Setup] Run Keywords Announce Message START TEST RwCoreFailAndRestart
- ... AND Start Logging RwCoreFailAndRestart
- ... AND Clear All Devices Then Create New Device
- [Teardown] Run Keywords Collect Logs
- ... AND Stop Logging RwCoreFailAndRestart
- ... AND Announce Message END TEST RwCoreFailAndRestart
- ... AND Delete Device and Verify
- Run Keyword and Ignore Error Collect Logs
- Run Keyword If ${has_dataplane} Clean Up Linux
- ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
- Set Global Variable ${of_id}
- FOR ${I} IN RANGE 0 ${num_onus}
- ${src}= Set Variable ${hosts.src[${I}]}
- ${dst}= Set Variable ${hosts.dst[${I}]}
- ${onu_device_id}= Get Device ID From SN ${src['onu']}
- ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
- ... ${of_id}
- # Bring up the device and verify it authenticates
- Wait Until Keyword Succeeds ${timeout} 5s Validate Device ENABLED ACTIVE REACHABLE
- ... ${onu_device_id} onu=True onu_reason=omci-flows-pushed
- Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${ONOS_SSH_IP}
- ... ${ONOS_SSH_PORT} ${onu_port}
- ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
- ... /tmp/wpa ${src['dp_iface_name']} log
- Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
- ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
- ... ${src['container_type']} ${src['container_name']} ${wpa_log}
- Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
- ... ${ONOS_SSH_PORT} ${onu_port}
- END
-
- # Scale down the rw-core deployment to 0 PODs and once confirmed, scale it back to 1
- Scale K8s Deployment voltha voltha-rw-core 0
- Wait Until Keyword Succeeds ${timeout} 2s Pod Does Not Exist voltha voltha-rw-core
- # Ensure the ofagent POD goes "not-ready" as expected
- Wait Until keyword Succeeds ${timeout} 2s
- ... Check Expected Available Deployment Replicas voltha voltha-ofagent 0
- # Scale up the core deployment and make sure both it and the ofagent deployment are back
- Scale K8s Deployment voltha voltha-rw-core 1
- Wait Until Keyword Succeeds ${timeout} 2s
- ... Check Expected Available Deployment Replicas voltha voltha-rw-core 1
- Wait Until Keyword Succeeds ${timeout} 2s
- ... Check Expected Available Deployment Replicas voltha voltha-ofagent 1
- # For some reason scaling down and up the POD behind a service causes the port forward to stop working,
- # so restart the port forwarding for the API service
- Restart VOLTHA Port Foward voltha-api-minimal
- # Ensure that the ofagent pod is up and ready and the device is available in ONOS, this
- # represents system connectivity being restored
- Wait Until Keyword Succeeds ${timeout} 2s Device Is Available In ONOS
- ... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT} ${of_id}
-
- FOR ${I} IN RANGE 0 ${num_onus}
- # Add subscriber access and verify that DHCP completes to ensure system is still functioning properly
- Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${ONOS_SSH_IP}
- ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
- Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
- ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
- ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
- ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
- ... ${dst['container_name']}
- Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
- ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
- END
-
Verify openolt adapter restart before subscriber provisioning
[Documentation] Deploys an device instance and waits for it to authenticate. After
... authentication is successful the rw-core deployment is scaled to 0 instances to
@@ -502,6 +431,77 @@
Run Keyword and Ignore Error Collect Logs
+Sanity E2E Test for OLT/ONU on POD With Core Fail and Restart
+ [Documentation] Deploys an device instance and waits for it to authenticate. After
+ ... authentication is successful the rw-core deployment is scaled to 0 instances to
+ ... simulate a POD crash. The test then scales the rw-core back to a single instance
+ ... and configures ONOS for access. The test succeeds if the device is able to
+ ... complete the DHCP sequence.
+ [Tags] functional rwcore-restart
+ [Setup] Run Keywords Announce Message START TEST RwCoreFailAndRestart
+ ... AND Start Logging RwCoreFailAndRestart
+ ... AND Clear All Devices Then Create New Device
+ [Teardown] Run Keywords Collect Logs
+ ... AND Stop Logging RwCoreFailAndRestart
+ ... AND Announce Message END TEST RwCoreFailAndRestart
+ ... AND Delete Device and Verify
+ Run Keyword and Ignore Error Collect Logs
+ Run Keyword If ${has_dataplane} Clean Up Linux
+ ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
+ Set Global Variable ${of_id}
+ FOR ${I} IN RANGE 0 ${num_onus}
+ ${src}= Set Variable ${hosts.src[${I}]}
+ ${dst}= Set Variable ${hosts.dst[${I}]}
+ ${onu_device_id}= Get Device ID From SN ${src['onu']}
+ ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
+ ... ${of_id}
+ # Bring up the device and verify it authenticates
+ Wait Until Keyword Succeeds ${timeout} 5s Validate Device ENABLED ACTIVE REACHABLE
+ ... ${onu_device_id} onu=True onu_reason=omci-flows-pushed
+ Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${ONOS_SSH_IP}
+ ... ${ONOS_SSH_PORT} ${onu_port}
+ ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
+ ... /tmp/wpa ${src['dp_iface_name']} log
+ Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
+ ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
+ ... ${src['container_type']} ${src['container_name']} ${wpa_log}
+ Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
+ ... ${ONOS_SSH_PORT} ${onu_port}
+ END
+
+ # Scale down the rw-core deployment to 0 PODs and once confirmed, scale it back to 1
+ Scale K8s Deployment voltha voltha-rw-core 0
+ Wait Until Keyword Succeeds ${timeout} 2s Pod Does Not Exist voltha voltha-rw-core
+ # Ensure the ofagent POD goes "not-ready" as expected
+ Wait Until keyword Succeeds ${timeout} 2s
+ ... Check Expected Available Deployment Replicas voltha voltha-ofagent 0
+ # Scale up the core deployment and make sure both it and the ofagent deployment are back
+ Scale K8s Deployment voltha voltha-rw-core 1
+ Wait Until Keyword Succeeds ${timeout} 2s
+ ... Check Expected Available Deployment Replicas voltha voltha-rw-core 1
+ Wait Until Keyword Succeeds ${timeout} 2s
+ ... Check Expected Available Deployment Replicas voltha voltha-ofagent 1
+ # For some reason scaling down and up the POD behind a service causes the port forward to stop working,
+ # so restart the port forwarding for the API service
+ Restart VOLTHA Port Foward voltha-api-minimal
+ # Ensure that the ofagent pod is up and ready and the device is available in ONOS, this
+ # represents system connectivity being restored
+ Wait Until Keyword Succeeds ${timeout} 2s Device Is Available In ONOS
+ ... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT} ${of_id}
+
+ FOR ${I} IN RANGE 0 ${num_onus}
+ # Add subscriber access and verify that DHCP completes to ensure system is still functioning properly
+ Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${ONOS_SSH_IP}
+ ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
+ Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
+ ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
+ ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
+ ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
+ ... ${dst['container_name']}
+ Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
+ ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
+ END
+
ONU Reboot
[Documentation] Reboot ONU and verify that ONU comes up properly
[Tags] VOL-1957 RebootONU notready