[VOL-4358] Waiting for the device to complete the reconciliation process before continuing
Change-Id: I641ee1b41251e810d91798b82685419564554cb4
diff --git a/libraries/onu_utilities.robot b/libraries/onu_utilities.robot
index 593a964..3db563a 100755
--- a/libraries/onu_utilities.robot
+++ b/libraries/onu_utilities.robot
@@ -114,6 +114,46 @@
# teardown is used as 'return' for result of Validate ONU Devices With Duration (used for ONUNegativeStateTests)
[Teardown] Run Keyword If "${KEYWORD STATUS}"=="FAIL" Set Suite Variable ${StateTestAllONUs} False
+Wait for ONU Adapter Reconcile
+ [Documentation] After the OpenONU adapter is restarted waits for all the ONUs to go through the reconciling process
+ ... till they reach the expect OpenStatus
+ [Arguments] ${oper_status}
+ # wait for the reconcile to complete
+ # - we check that the first ONU state is set to reconciling. All of the ONUs will be set to reconciling
+ # very quickly at that point
+ # we need the Ignore Error to support he kafka case, in which the reconcile happens meanwhile we're waiting for
+ # the adapter to restart
+ Run Keyword And Ignore Error Wait Until Keyword Succeeds ${timeout}
+ ... 100ms Check ONU OperStatus ${hosts.src[0]['onu']} RECONCILING
+ # - then we wait that all ONU move to the next state
+ ${onu_list} Create List
+ FOR ${I} IN RANGE 0 ${num_all_onus}
+ ${onu_sn}= Set Variable ${hosts.src[${I}]['onu']}
+ # skip the check if we have already performed it for this ONU
+ ${onu_id}= Get Index From List ${onu_list} ${onu_sn}
+ Continue For Loop If -1 != ${onu_id}
+ Append To List ${onu_list} ${onu_sn}
+ Wait Until Keyword Succeeds ${timeout} 100ms Check ONU OperStatus ${onu_sn} ${oper_status}
+ END
+
+Check ONU OperStatus
+ [Documentation] Checks that all ONUs OperStatus is ACTIVE
+ [Arguments] ${sn} ${oper_status}
+ ${List_ONU_Serial} Create List
+ Build ONU SN List ${List_ONU_Serial}
+ ${cmd}= Catenate voltctl -c ${VOLTCTL_CONFIG} device list -m 8MB -f Type=brcm_openomci_onu
+ ... --format "{{.SerialNumber}}\t{{.AdminState}}\t{{.OperStatus}}\t{{.ConnectStatus}}\t{{.Reason}}"
+ ... | grep -v SERIALNUMBER | grep ${sn}
+ ${rc} ${output}= Run and Return Rc and Output ${cmd}
+ Should Be Equal As Integers ${rc} 0
+ Log ${output}
+
+ @{words}= Split String ${output} \t
+ ${sn}= Set Variable ${words[0]}
+ ${opstatus}= Set Variable ${words[2]}
+ Log OperStatus is ${opstatus} for device ${sn}
+ Should Be True '${opstatus}' == '${oper_status}' OperStatus ${oper_status} not matched on ONU ${sn}
+
Log Ports
[Documentation] This keyword logs all port data available in ONOS of first port per ONU
[Arguments] ${onlyenabled}=False
diff --git a/libraries/voltctl.robot b/libraries/voltctl.robot
index e926c0d..629259b 100755
--- a/libraries/voltctl.robot
+++ b/libraries/voltctl.robot
@@ -42,7 +42,7 @@
${rc} ${device_id}= Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} device create -t ${type} -H ${ip}:${port}
Log ${device_id}
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Failed to Create Device beause of ${device_id}
[Return] ${device_id}
Enable Device
@@ -50,28 +50,28 @@
[Documentation] Enables a device in VOLTHA
${rc} ${output}= Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} device enable ${device_id}
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Failed to Enable Device beause of ${output}
Disable Device
[Arguments] ${device_id}
[Documentation] Disables a device in VOLTHA
${rc} ${output}= Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} device disable ${device_id}
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Failed to Disable Device beause of ${output}
Delete Device
[Arguments] ${device_id}
[Documentation] Deletes a device in VOLTHA
${rc} ${output}= Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} device delete ${device_id}
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Failed to Delete Device beause of ${output}
Reboot Device
[Arguments] ${device_id}
[Documentation] Reboot the OLT using voltctl command
${rc} ${output}= Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} device reboot ${device_id}
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Failed to Reboot Device beause of ${output}
Disable Devices In Voltha
[Documentation] Disables all the known devices in voltha
@@ -83,7 +83,8 @@
Should Be Equal As Integers ${rc} 0
${rc} ${output}= Run Keyword If len('${devices}') != 0 Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} device disable ${devices}
- Run Keyword If len('${devices}') != 0 Should Be Equal As Integers ${rc} 0
+ Run Keyword If len('${devices}') != 0
+ ... Should Be Equal As Integers ${rc} 0 Failed to Disable Devices beause of ${output}
Test Devices Disabled In Voltha
[Documentation] Tests to verify that all devices in VOLTHA are disabled
@@ -329,7 +330,8 @@
${cmd}= Catenate voltctl -c ${VOLTCTL_CONFIG} device list -m 8MB -f Type=brcm_openomci_onu
... --format "{{.SerialNumber}}\t{{.AdminState}}\t{{.OperStatus}}\t{{.ConnectStatus}}\t{{.Reason}}" | grep -v SERIALNUMBER
${rc} ${output}= Run and Return Rc and Output ${cmd}
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Failed Validate ONU Devices because of ${output}
+ Log ${output}
${timeCurrent} = Get Current Date
${timeTotalMs} = Subtract Date From Date ${timeCurrent} ${startTime} result_format=number
@{Results}= Split String ${output} \n
diff --git a/tests/openonu-go-adapter/Voltha_ONUReconcileTests.robot b/tests/openonu-go-adapter/Voltha_ONUReconcileTests.robot
index 4198f98..678365d 100755
--- a/tests/openonu-go-adapter/Voltha_ONUReconcileTests.robot
+++ b/tests/openonu-go-adapter/Voltha_ONUReconcileTests.robot
@@ -265,6 +265,7 @@
Current State Test All Onus ${expected_onu_reason}
Run Keyword If ${usekill2restart} Kill And Check Onu Adaptor ${NAMESPACE}
... ELSE Restart And Check Onu Adaptor ${NAMESPACE}
+ Wait for ONU Adapter Reconcile ACTIVE
Run Keyword If "${workflow}"=="DT" Perform Sanity Test DT
... ELSE IF "${workflow}"=="TT" Perform Sanity Tests TT
... ELSE Perform Sanity Test
@@ -300,6 +301,7 @@
... ELSE Current State Test All Onus omci-admin-lock alternativeonustate=${alternativeonustates}
Run Keyword If ${usekill2restart} Kill And Check Onu Adaptor ${NAMESPACE}
... ELSE Restart And Check Onu Adaptor ${NAMESPACE}
+ Wait for ONU Adapter Reconcile UNKNOWN
Run Keyword If "${workflow}"=="DT" Current State Test All Onus omci-admin-lock
... ELSE IF "${workflow}"=="TT" Current State Test All Onus omci-admin-lock
... ELSE Current State Test All Onus omci-admin-lock alternativeonustate=${alternativeonustates}
@@ -334,6 +336,7 @@
... ELSE Perform Sanity Test
Run Keyword If ${usekill2restart} Kill And Check Onu Adaptor ${NAMESPACE}
... ELSE Restart And Check Onu Adaptor ${NAMESPACE}
+ Wait for ONU Adapter Reconcile ACTIVE
Run Keyword If "${workflow}"=="DT" Perform Sanity Test DT ${suppressaddsubscriber}
... ELSE IF "${workflow}"=="TT" Perform Sanity Tests TT ${suppressaddsubscriber}
... ELSE Perform Sanity Test ${suppressaddsubscriber}