[VOL-4328] Added voltha-component restart with continuous ping in background scenarios for TT
Change-Id: Ib44da13f094aa11321b14f0883df01ede4c02665
diff --git a/tests/tt-workflow/Voltha_TT_FailureScenarios.robot b/tests/tt-workflow/Voltha_TT_FailureScenarios.robot
index 46ba95d..c1d4885 100755
--- a/tests/tt-workflow/Voltha_TT_FailureScenarios.robot
+++ b/tests/tt-workflow/Voltha_TT_FailureScenarios.robot
@@ -480,6 +480,124 @@
END
Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Tests TT
+Verify restart openonu-adapter container while continuously running ping in background for TT
+ [Documentation] Restart openonu-adapter container after VOLTHA is operational.
+ ... Run the ping continuously in background during container restart,
+ ... and verify that there should be no affect on the dataplane.
+ [Tags] functionalTT Restart-OpenOnu-Ping-TT non-critical
+ [Setup] Start Logging Restart-OpenOnu-Ping-TT
+ [Teardown] Run Keywords Collect Logs
+ ... AND Stop Logging Restart-OpenOnu-Ping-TT
+ Clear All Devices Then Create New Device
+ # Performing Sanity Test to make sure subscribers are all DHCP and pingable
+ Run Keyword If ${has_dataplane} Clean Up Linux
+ Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test TT
+ FOR ${I} IN RANGE 0 ${num_all_onus}
+ ${src}= Set Variable ${hosts.src[${I}]}
+ ${dst}= Set Variable ${hosts.dst[${I}]}
+ ${service_type}= Get Variable Value ${src['service_type']} "null"
+ Continue For Loop If '${service_type}' == 'mcast'
+ ${ping_output_file}= Set Variable /tmp/${src['onu']}_${service_type}_ping
+ Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
+ ... Wait Until Keyword Succeeds 60s 2s
+ ... Run Ping In Background ${ping_output_file} ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
+ ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
+ END
+ ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
+ Log ${podStatusOutput}
+ ${countBeforeRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
+ ${podName} Set Variable adapter-open-onu
+ Wait Until Keyword Succeeds ${timeout} 15s Delete K8s Pods By Label ${NAMESPACE} app ${podName}
+ Sleep 5s
+ Wait Until Keyword Succeeds ${timeout} 2s Validate Pods Status By Label ${NAMESPACE}
+ ... app ${podName} Running
+ Wait Until Keyword Succeeds ${timeout} 3s Pods Are Ready By Label ${NAMESPACE} app ${podName}
+ ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
+ Log ${podStatusOutput}
+ ${countAfterRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
+ Should Be Equal As Strings ${countAfterRestart} ${countBeforeRestart}
+ FOR ${I} IN RANGE 0 ${num_all_onus}
+ ${src}= Set Variable ${hosts.src[${I}]}
+ ${dst}= Set Variable ${hosts.dst[${I}]}
+ ${service_type}= Get Variable Value ${src['service_type']} "null"
+ Continue For Loop If '${service_type}' == 'mcast'
+ Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
+ ... Wait Until Keyword Succeeds 60s 2s
+ ... Stop Ping Running In Background ${src['ip']} ${src['user']} ${src['pass']}
+ ... ${src['container_type']} ${src['container_name']}
+ END
+ FOR ${I} IN RANGE 0 ${num_all_onus}
+ ${src}= Set Variable ${hosts.src[${I}]}
+ ${dst}= Set Variable ${hosts.dst[${I}]}
+ ${service_type}= Get Variable Value ${src['service_type']} "null"
+ Continue For Loop If '${service_type}' == 'mcast'
+ ${ping_output_file}= Set Variable /tmp/${src['onu']}_${service_type}_ping
+ ${ping_output}= Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
+ ... Wait Until Keyword Succeeds 60s 2s
+ ... Retrieve Remote File Contents ${ping_output_file} ${src['ip']} ${src['user']} ${src['pass']}
+ ... ${src['container_type']} ${src['container_name']}
+ Check Ping Result True ${ping_output}
+ END
+
+Verify restart openolt-adapter container while continuously running ping in background for TT
+ [Documentation] Restart openolt-adapter container after VOLTHA is operational.
+ ... Run the ping continuously in background during container restart,
+ ... and verify that there should be no affect on the dataplane.
+ [Tags] functionalTT Restart-OpenOlt-Ping-TT non-critical
+ [Setup] Start Logging Restart-OpenOlt-Ping-TT
+ [Teardown] Run Keywords Collect Logs
+ ... AND Stop Logging Restart-OpenOlt-Ping-TT
+ Clear All Devices Then Create New Device
+ # Performing Sanity Test to make sure subscribers are all DHCP and pingable
+ Run Keyword If ${has_dataplane} Clean Up Linux
+ Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test TT
+ FOR ${I} IN RANGE 0 ${num_all_onus}
+ ${src}= Set Variable ${hosts.src[${I}]}
+ ${dst}= Set Variable ${hosts.dst[${I}]}
+ ${service_type}= Get Variable Value ${src['service_type']} "null"
+ Continue For Loop If '${service_type}' == 'mcast'
+ ${ping_output_file}= Set Variable /tmp/${src['onu']}_${service_type}_ping
+ Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
+ ... Wait Until Keyword Succeeds 60s 2s
+ ... Run Ping In Background ${ping_output_file} ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
+ ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
+ END
+ ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
+ Log ${podStatusOutput}
+ ${countBeforeRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
+ ${podName} Set Variable ${OLT_ADAPTER_APP_LABEL}
+ Wait Until Keyword Succeeds ${timeout} 15s Delete K8s Pods By Label ${NAMESPACE} app ${podName}
+ Sleep 5s
+ Wait Until Keyword Succeeds ${timeout} 2s Validate Pods Status By Label ${NAMESPACE}
+ ... app ${podName} Running
+ Wait Until Keyword Succeeds ${timeout} 3s Pods Are Ready By Label ${NAMESPACE} app ${podName}
+ ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
+ Log ${podStatusOutput}
+ ${countAfterRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
+ Should Be Equal As Strings ${countAfterRestart} ${countBeforeRestart}
+ FOR ${I} IN RANGE 0 ${num_all_onus}
+ ${src}= Set Variable ${hosts.src[${I}]}
+ ${dst}= Set Variable ${hosts.dst[${I}]}
+ ${service_type}= Get Variable Value ${src['service_type']} "null"
+ Continue For Loop If '${service_type}' == 'mcast'
+ Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
+ ... Wait Until Keyword Succeeds 60s 2s
+ ... Stop Ping Running In Background ${src['ip']} ${src['user']} ${src['pass']}
+ ... ${src['container_type']} ${src['container_name']}
+ END
+ FOR ${I} IN RANGE 0 ${num_all_onus}
+ ${src}= Set Variable ${hosts.src[${I}]}
+ ${dst}= Set Variable ${hosts.dst[${I}]}
+ ${service_type}= Get Variable Value ${src['service_type']} "null"
+ Continue For Loop If '${service_type}' == 'mcast'
+ ${ping_output_file}= Set Variable /tmp/${src['onu']}_${service_type}_ping
+ ${ping_output}= Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
+ ... Wait Until Keyword Succeeds 60s 2s
+ ... Retrieve Remote File Contents ${ping_output_file} ${src['ip']} ${src['user']} ${src['pass']}
+ ... ${src['container_type']} ${src['container_name']}
+ Check Ping Result True ${ping_output}
+ END
+
*** Keywords ***
Setup Suite
[Documentation] Set up the test suite