[VOL-1959] Failure scenario: test ONU adapter crash
Change-Id: I55cc222d1b007a7eae5b6258b3130570db6d6c90
diff --git a/libraries/k8s.robot b/libraries/k8s.robot
index 781d0b2..62a5b11 100644
--- a/libraries/k8s.robot
+++ b/libraries/k8s.robot
@@ -52,6 +52,20 @@
${rc} ${output}= Run and Return Rc and Output kubectl delete pod ${restart_pod_name} -n ${namespace} --grace-period=0 --force
Log ${output}
+Validate Pod Status
+ [Arguments] ${pod_name} ${namespace} ${expectedStatus}
+ [Documentation] To run the kubectl command and check the status of the given pod matches the expected status
+ ${length}= Run ${KUBECTL_CONFIG}; kubectl get pod -n ${namespace} | wc -l
+ FOR ${index} IN RANGE ${length}-1
+ ${currentPodName}= Run ${KUBECTL_CONFIG}; kubectl get pod -n ${namespace} -o jsonpath={.items[${index}].status.containerStatuses[0].name}
+ Log Required Pod : ${pod_name}
+ Log Current Pod: ${currentPodName}
+ Run Keyword and Ignore Error Run Keyword If '${currentPodName}'=='${pod_name}' Exit For Loop
+ END
+ ${currentStatusofPod}= Run ${KUBECTL_CONFIG}; kubectl get pod -n ${namespace} -o jsonpath={.items[${index}].status.phase}
+ Log ${currentStatusofPod}
+ Should Contain ${currentStatusofPod} ${expectedStatus}
+
Verify All Voltha Pods For Any Error Logs
[Arguments] ${datetime}
[Documentation] This keyword checks for the error occurence in the voltha pods
diff --git a/libraries/utils.robot b/libraries/utils.robot
index cc7e1df..c235383 100644
--- a/libraries/utils.robot
+++ b/libraries/utils.robot
@@ -126,3 +126,11 @@
Log ${result}
#Should Contain ${result} DHCPRELEASE
[Return] ${result}
+
+Check Remote File Contents For WPA Logs
+ [Arguments] ${file_should_exist} ${file} ${pattern} ${ip} ${user} ${pass}=${None} ${container_type}=${None}
+ ... ${container_name}=${None} ${prompt}=~$
+ [Documentation] Checks for particular pattern count in a file
+ ${result}= Login And Run Command On Remote System cat ${file} | grep '${pattern}' | wc -l ${ip} ${user} ${pass}
+ ... ${container_type} ${container_name} ${prompt}
+ [Return] ${result}
diff --git a/tests/functional/Voltha_PODTests.robot b/tests/functional/Voltha_PODTests.robot
index cf5bec7..c1ccc7d 100644
--- a/tests/functional/Voltha_PODTests.robot
+++ b/tests/functional/Voltha_PODTests.robot
@@ -212,6 +212,51 @@
END
Run Keyword and Ignore Error Collect Logs
+Check ONU adapter crash not forcing authentication again
+ [Documentation] After ONU adapter restart, checks wpa log for 'authentication started'
+ ... message count to make sure auth not started again and validates EAP status and ping.
+ ... Assuming that test1 or sanity was executed where all the ONUs are authenticated/DHCP/pingable
+ [Tags] functional ONUAdaptCrash
+ [Setup] None
+ [Teardown] None
+ @{before_list}= Create List
+ @{after_list}= Create List
+ FOR ${I} IN RANGE 0 ${num_onus}
+ ${src}= Set Variable ${hosts.src[${I}]}
+ ${dst}= Set Variable ${hosts.dst[${I}]}
+ ${onu_device_id}= Get Device ID From SN ${src['onu']}
+ ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
+ ... ${of_id}
+ ${before}= Run Keyword If ${has_dataplane} Check Remote File Contents For WPA Logs
+ ... True /tmp/wpa.log authentication started ${src['ip']} ${src['user']} ${src['pass']}
+ ... ${src['container_type']} ${src['container_name']}
+ Append To List ${before_list} ${before}
+ END
+ Wait Until Keyword Succeeds ${timeout} 15s Restart Pod ${NAMESPACE} adapter-open-onu
+ Wait Until Keyword Succeeds ${timeout} 2s Validate Pod Status ${podName} ${NAMESPACE}
+ ... Running
+ FOR ${I} IN RANGE 0 ${num_onus}
+ ${src}= Set Variable ${hosts.src[${I}]}
+ ${dst}= Set Variable ${hosts.dst[${I}]}
+ ${onu_device_id}= Get Device ID From SN ${src['onu']}
+ ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
+ ... ${of_id}
+ ${after}= Run Keyword If ${has_dataplane} Check Remote File Contents For WPA Logs
+ ... True /tmp/wpa.log authentication started ${src['ip']} ${src['user']} ${src['pass']}
+ ... ${src['container_type']} ${src['container_name']}
+ Append To List ${after_list} ${after}
+ ${output}= Run Keyword If ${has_dataplane} Login And Run Command On Remote System
+ ... wpa_cli status | grep SUCCESS ${src['ip']} ${src['user']} ${src['pass']}
+ ... ${src['container_type']} ${src['container_name']}
+ Should Contain ${output} SUCCESS
+ Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Wait Until Keyword Succeeds 60s 2s Check Ping
+ ... True ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']} ${src['ip']} ${src['user']}
+ ... ${src['pass']} ${src['container_type']} ${src['container_name']}
+ END
+ Lists Should Be Equal ${after_list} ${before_list}
+ Log ${after_list}
+ Log ${before_list}
+ Run Keyword and Ignore Error Collect Logs
Test Disable and Enable ONU scenario for ATT workflow
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario: