[VOL-4115] Added more scenarios to DT Soak testing, and logging flag to enable/disable Collect Logs
Change-Id: Ieecd5aa63716f38a84d845274b4657f679e6c3e1
diff --git a/tests/dt-workflow/Voltha_DT_FailureScenarios.robot b/tests/dt-workflow/Voltha_DT_FailureScenarios.robot
index a6d2bf7..355136d 100755
--- a/tests/dt-workflow/Voltha_DT_FailureScenarios.robot
+++ b/tests/dt-workflow/Voltha_DT_FailureScenarios.robot
@@ -55,6 +55,10 @@
# Per-test logging on failure is turned off by default; set this variable to enable
${container_log_dir} ${None}
+# logging flag to enable Collect Logs, can be passed via the command line too
+# example: -v logging:False
+${logging} True
+
# Flag specific to Soak Jobs
${SOAK_TEST} False
@@ -66,7 +70,7 @@
... controls the power off/on ONUs/OLT remotely (simulating a physical reboot)
[Tags] functionalDt PowerSwitchOnuRebootDt VOL-2819 PowerSwitch
[Setup] Start Logging RebootOnu_PowerSwitch_Dt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging RebootOnu_PowerSwitch_Dt
... AND Delete All Devices and Verify
# Add OLT device
@@ -129,7 +133,7 @@
... Test performs a physical reboot, performs "reboot" from the OLT CLI
[Tags] functionalDt PhysicalOltRebootDt VOL-2817
[Setup] Start Logging RebootOlt_Physical_Dt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging RebootOlt_Physical_Dt
... AND Delete All Devices and Verify
# Add OLT device
@@ -176,13 +180,12 @@
Verify restart openonu-adapter container after subscriber provisioning for DT
[Documentation] Restart openonu-adapter container after VOLTHA is operational.
... Prerequisite : ONUs are authenticated and pingable.
- [Tags] functionalDt Restart-OpenOnu-Dt
+ [Tags] functionalDt Restart-OpenOnu-Dt soak
[Setup] Start Logging Restart-OpenOnu-Dt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging Restart-OpenOnu-Dt
- ... AND Delete All Devices and Verify
# Add OLT device
- Setup
+ Run Keyword If 'SOAK_TEST'=='False' Setup
# Performing Sanity Test to make sure subscribers are all DHCP and pingable
Run Keyword If ${has_dataplane} Clean Up Linux
Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test DT
@@ -202,16 +205,17 @@
${countAfterRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
Should Be Equal As Strings ${countAfterRestart} ${countBeforeRestart}
Log to console Pod ${podName} restarted and sanity checks passed successfully
+ Run Keyword If 'SOAK_TEST'=='False' Delete All Devices and Verify
Verify restart openolt-adapter container after subscriber provisioning for DT
[Documentation] Restart openolt-adapter container after VOLTHA is operational.
... Prerequisite : ONUs are authenticated and pingable.
- [Tags] functionalDt Restart-OpenOlt-Dt
+ [Tags] functionalDt Restart-OpenOlt-Dt soak
[Setup] Start Logging Restart-OpenOlt-Dt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging Restart-OpenOlt-Dt
# Add OLT device
- setup
+ Run Keyword If 'SOAK_TEST'=='False' setup
# Performing Sanity Test to make sure subscribers are all DHCP and pingable
Run Keyword If ${has_dataplane} Clean Up Linux
Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test DT
@@ -241,7 +245,7 @@
[Tags] functionalDt olt-adapter-restart-Dt soak
[Setup] Start Logging OltAdapterRestart-Dt
#... AND Clear All Devices Then Create New Device
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging OltAdapterRestart-Dt
# Add OLT and perform sanity test
#setup
@@ -304,7 +308,7 @@
... Prerequisite : ONUs are authenticated and pingable.
[Tags] functionalDt ofagentRestart-Dt soak
[Setup] Start Logging ofagentRestart-Dt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging ofagentRestart-Dt
... AND Scale K8s Deployment ${NAMESPACE} voltha-voltha-ofagent 1
# set timeout value
@@ -365,10 +369,10 @@
... simulate a POD crash. The test then scales the rw-core back to a single instance
... and configures ONOS for access. The test succeeds if the device is able to
... complete the DHCP sequence.
- [Tags] functionalDt rwcore-restart-Dt soak
+ [Tags] functionalDt rwcore-restart-Dt
[Setup] Run Keywords Start Logging RwCoreFailAndRestart-Dt
... AND Clear All Devices Then Create New Device
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging RwCoreFailAndRestart-Dt
#... AND Delete Device and Verify
Run Keyword If ${has_dataplane} Clean Up Linux
diff --git a/tests/dt-workflow/Voltha_DT_PODTests.robot b/tests/dt-workflow/Voltha_DT_PODTests.robot
index 7f03385..378b673 100644
--- a/tests/dt-workflow/Voltha_DT_PODTests.robot
+++ b/tests/dt-workflow/Voltha_DT_PODTests.robot
@@ -63,6 +63,10 @@
# Per-test logging on failure is turned off by default; set this variable to enable
${container_log_dir} ${None}
+# logging flag to enable Collect Logs, can be passed via the command line too
+# example: -v logging:False
+${logging} True
+
# Flag specific to Soak Jobs
${SOAK_TEST} False
${bbsim_port} 50060
@@ -74,7 +78,7 @@
... controls the power off/on ONUs/OLT remotely (simulating a physical reboot)
[Tags] functionalDt PowerSwitch RebootAllDTONUs soak
[Setup] Start Logging RebootAllDTONUs
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging RebootAllDTONUs
Power Switch Connection Suite ${web_power_switch.ip} ${web_power_switch.user} ${web_power_switch.password}
FOR ${I} IN RANGE 0 ${num_all_onus}
@@ -91,7 +95,7 @@
... but only to mock the load on Soak POD.
[Tags] soak
[Setup] Start Logging soakPodCreateBBSimLoad
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging soakPodCreateBBSimLoad
${num_bbsim} Get Length ${bbsim}
FOR ${I} IN RANGE 0 ${num_bbsim}
@@ -119,7 +123,7 @@
... Inner vlans from the RG should not change
[Tags] sanityDt soak
[Setup] Start Logging SanityTestDt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging SanityTestDt
Setup ${SOAK_TEST}
Run Keyword If ${has_dataplane} Clean Up Linux
@@ -133,7 +137,7 @@
... Re-add the subscriber, and validate that the flows are present and pings are successful
[Tags] functionalDt SubAddDeleteDt soak
[Setup] Start Logging SubAddDeleteDt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging SubAddDeleteDt
FOR ${I} IN RANGE 0 ${num_all_onus}
${src}= Set Variable ${hosts.src[${I}]}
@@ -200,7 +204,7 @@
... Perform enable on the ONUs and validate that the pings are successful
[Tags] functionalDt DisableEnableONUDt soak
[Setup] Start Logging DisableEnableONUDt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging DisableEnableONUDt
FOR ${I} IN RANGE 0 ${num_all_onus}
${src}= Set Variable ${hosts.src[${I}]}
@@ -239,9 +243,9 @@
... Assuming that all the ONUs are DHCP/pingable (i.e. assuming sanityDt test was executed)
... Perform disable on the OLT and validate ONUs state and that the pings do not succeed
... Perform delete on the OLT, Re-do Setup (Recreate the OLT) and Perform Sanity Test DT
- [Tags] functionalDt DisableDeleteOLTDt
+ [Tags] functionalDt DisableDeleteOLTDt soak
[Setup] Start Logging DisableDeleteOLTDt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging DisableDeleteOLTDt
# Disable and Validate OLT Device
FOR ${I} IN RANGE 0 ${olt_count}
@@ -288,7 +292,7 @@
... Verify Device Flows Removed ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
END
# Re-do Setup (Recreate the OLT) and Perform Sanity Test DT
- Run Keyword Setup
+ Run Keyword Setup ${SOAK_TEST}
Run Keyword If ${has_dataplane} Clean Up Linux
Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test DT
#Run Keyword If ${has_dataplane} Clean Up Linux
@@ -300,7 +304,7 @@
... Perform enable on the OLT and validate that the pings are successful
[Tags] functionalDt DisableEnableOLTDt soak
[Setup] Start Logging DisableEnableOLTDt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging DisableEnableOLTDt
# Disable and Validate OLT Device
FOR ${I} IN RANGE 0 ${olt_count}
@@ -355,9 +359,9 @@
... Disable and Delete the OLT
... Create/Enable the same OLT again
... Validate DHCP/E2E pings succeed for all the ONUs connected to the OLT
- [Tags] functionalDt DeleteReAddOLTDt
+ [Tags] functionalDt DeleteReAddOLTDt soak
[Setup] Start Logging DeleteReAddOLTDt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging DeleteReAddOLTDt
FOR ${I} IN RANGE 0 ${olt_count}
${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
@@ -367,7 +371,7 @@
... Verify Device Flows Removed ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
END
# Recreate the OLTs
- Setup
+ Setup ${SOAK_TEST}
Run Keyword If ${has_dataplane} Clean Up Linux
Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test DT
@@ -378,7 +382,7 @@
... so calling setup at the end to add the devices back to avoid the confusion.
[Tags] functionalDt DisableDeleteONUOLTDt
[Setup] Start Logging DisableDeleteONUOLTDt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging DisableDeleteONUOLTDt
@{onu_reason}= Create List initial-mib-downloaded omci-flows-pushed
FOR ${I} IN RANGE 0 ${num_all_onus}
@@ -441,7 +445,7 @@
... Assumes iperf3 and jq installed on client and iperf -s running on DHCP server
[Tags] dataplaneDt BandwidthProfileTCPDt VOL-3061 soakDataplane
[Setup] Start Logging BandwidthProfileTCPDt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging BandwidthProfileTCPDt
Pass Execution If '${has_dataplane}'=='False' Bandwidth profile validation can be done only in
... physical pod. Skipping this test in BBSIM.
@@ -502,7 +506,7 @@
... Assumes iperf3 and jq installed on client and iperf -s running on DHCP server
[Tags] dataplaneDt BandwidthProfileUDPDt VOL-3061 soakDataplane
[Setup] Start Logging BandwidthProfileUDPDt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging BandwidthProfileUDPDt
Pass Execution If '${has_dataplane}'=='False' Bandwidth profile validation can be done only in
... physical pod. Skipping this test in BBSIM.
@@ -573,7 +577,7 @@
... the pbits are preserved by the PON.
[Tags] dataplaneDt TechProfileDt VOL-3291 soakDataplane
[Setup] Start Logging TechProfileDt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging TechProfileDt
Pass Execution If '${has_dataplane}'=='False'
... Skipping test: Technology profile validation can be done only in physical pod
@@ -618,9 +622,9 @@
... Assuming that all the ONUs are DHCP/pingable (i.e. assuming sanityDt test was executed)
... Perform disable on the OLT PON Port and validate that the pings do not succeed
... Perform enable on the OLT PON Port and validate that the pings are successful
- [Tags] functionalDt DisableEnableOltPonPortDt VOL-2577
+ [Tags] functionalDt DisableEnableOltPonPortDt VOL-2577 soak
[Setup] Start Logging DisableEnableOltPonPortDt
- [Teardown] Run Keywords Collect Logs
+ [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
... AND Stop Logging DisableEnableOltPonPortDt
FOR ${I} IN RANGE 0 ${olt_count}
${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn