Add more explicit console logging + minor fixes
* The checks with keywords such as "Should Not Be Empty", "Should
Be Equal As"... should have a more explicit message on fail, so that
the console output shows a clearer failing reason.
* Other minor fixes included in the commit are:
* The Sadis file comparison to None string.
* The KUBECTL_CONFIG variable not properly set, altough
apparently unused.
Change-Id: I9305ba791dd43c9088a7bcbcc9af568ae127f60f
diff --git a/tests/dmi-interface/dmi-hw-management.robot b/tests/dmi-interface/dmi-hw-management.robot
index a345426..7c3fd2f 100755
--- a/tests/dmi-interface/dmi-hw-management.robot
+++ b/tests/dmi-interface/dmi-hw-management.robot
@@ -179,7 +179,7 @@
Log ${element}
${component_uuid}= utility.get_uuid_from_Inventory_Element ${element} ${component_name}
END
- Should Not Be Equal ${None} ${component_uuid}
+ Should Not Be Equal ${None} ${component_uuid} Could not find UUID for ${component_name}
[Return] ${component_uuid}
Loggable Entities
diff --git a/tests/functional/K8S_SystemTest.robot b/tests/functional/K8S_SystemTest.robot
index ba14607..47d2b2b 100644
--- a/tests/functional/K8S_SystemTest.robot
+++ b/tests/functional/K8S_SystemTest.robot
@@ -98,7 +98,7 @@
[Documentation] Get the number of configured ETCD nodes
${rc} ${size}= Run and Return Rc and Output
... kubectl -n ${namespace} get ${ETCD_resources} ${ETCD_name} -o jsonpath='{.status.replicas}'
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Could not get ETCD replica count
[Return] ${size}
Scale ETCD
@@ -106,16 +106,16 @@
[Documentation] Scale down the number of ETCD pod
${rc}= Run and Return Rc
... kubectl -n ${namespace} patch ${ETCD_resources} ${ETCD_name} -p '{"spec":{"replicas": ${size}}}'
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Could not scale ETCD
Validate ETCD Size
[Arguments] ${namespace} ${ETCD_cluster_size}
[Documentation] Scale down the number of ETCD pod
${rc} ${size}= Run and Return Rc and Output
... kubectl -n ${namespace} get ${ETCD_resources} ${ETCD_name} -o jsonpath='{.status.replicas}'
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Could not get ETCD replica count
Should Be Equal As Integers ${size} ${ETCD_cluster_size} Unexpected number of replicas
${rc} ${size}= Run and Return Rc and Output
... kubectl -n ${namespace} get ${ETCD_resources} ${ETCD_name} -o jsonpath='{.status.readyReplicas}'
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Could not get ETC ready replica count
Should Be Equal As Integers ${size} ${ETCD_cluster_size} Unexpected number of ready replicas
diff --git a/tests/functional/Voltha_ErrorScenarios.robot b/tests/functional/Voltha_ErrorScenarios.robot
index e617403..c0e6301 100755
--- a/tests/functional/Voltha_ErrorScenarios.robot
+++ b/tests/functional/Voltha_ErrorScenarios.robot
@@ -113,7 +113,7 @@
[Teardown] Run Keywords Collect Logs
... AND Stop Logging DisableInvalidDevice
${rc} ${output}= Run and Return Rc and Output voltctl -c ${VOLTCTL_CONFIG} device list -o json
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Could not get device list
${jsondata}= To Json ${output}
Log ${jsondata}
${length}= Get Length ${jsondata}
@@ -127,7 +127,7 @@
${fakeDeviceId} Replace String Using Regexp ${device_id} \\d\\d xx count=1
Log ${fakeDeviceId}
#Ensure that the new id created is not in the device id list
- List Should Not Contain Value ${ids} ${fakeDeviceId}
+ List Should Not Contain Value ${ids} ${fakeDeviceId} Device list should not contain ${fakeDeviceID}, but it does
#Disable fake device id
${rc} ${output}= Run and Return Rc and Output voltctl -c ${VOLTCTL_CONFIG} device disable ${fakeDeviceId}
Should Contain ${output} Error while disabling '${fakeDeviceId}'
@@ -263,10 +263,10 @@
#Check whether logical devices are also created
${rc} ${output}= Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} logicaldevice list
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Could not get logical device list
Log ${output}
${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
- Should Not Be Empty ${logical_id}
+ Should Not Be Empty ${logical_id} Could not get logical device ID in VOLTHA for OLT serial number ${olt_serial_number}
${rc} ${output}= Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} logicaldevice disable ${logical_id}
Should Not Be Equal As Integers ${rc} 0
@@ -297,7 +297,7 @@
${olt_serial_number}= Get From Dictionary ${list_olts}[${I}] sn
#${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
- Should Be Empty ${logical_id}
+ Should Be Empty ${logical_id} Logical device ID is not empty
Run Keyword If ${has_dataplane} Sleep 180s
... ELSE Sleep 10s
${olt_device_id}= Run Keyword If "${list_olts}[${I}][type]" == "${None}"
@@ -310,10 +310,10 @@
Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Device ENABLED ACTIVE REACHABLE
... ${olt_serial_number}
${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
- Should Not Be Empty ${logical_id}
+ Should Not Be Empty ${logical_id} Could not get logical device ID in VOLTHA for OLT serial number ${olt_serial_number}
${rc} ${output}= Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} logicaldevice list
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Could not get logical device list
Log ${output}
Should Contain ${output} ${olt_device_id}
Set Suite Variable ${logical_id}
diff --git a/tests/functional/Voltha_PODTests.robot b/tests/functional/Voltha_PODTests.robot
index ddc687e..260fc94 100755
--- a/tests/functional/Voltha_PODTests.robot
+++ b/tests/functional/Voltha_PODTests.robot
@@ -301,7 +301,7 @@
${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
${rc} ${output}= Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} device disable ${olt_device_id}
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Could not disable device ${olt_device_id}
Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Device DISABLED UNKNOWN REACHABLE
... ${olt_serial_number}
END
@@ -430,7 +430,7 @@
... REACHABLE ${src['olt']}
${rc} ${output}= Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} device disable ${onu_device_id}
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Could not disable device ${olt_device_id}
Wait Until Keyword Succeeds ${timeout} 5s
... Validate Device DISABLED UNKNOWN
... REACHABLE ${src['onu']} onu=false
@@ -445,7 +445,7 @@
${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
${rc} ${output}= Run and Return Rc and Output
... voltctl -c ${VOLTCTL_CONFIG} device disable ${olt_device_id}
- Should Be Equal As Integers ${rc} 0
+ Should Be Equal As Integers ${rc} 0 Could not disable device ${olt_device_id}
Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Device DISABLED UNKNOWN REACHABLE
... ${olt_serial_number}
END
diff --git a/tests/openonu-go-adapter/Voltha_ONUFlowChecks.robot b/tests/openonu-go-adapter/Voltha_ONUFlowChecks.robot
index 1947f52..197efde 100755
--- a/tests/openonu-go-adapter/Voltha_ONUFlowChecks.robot
+++ b/tests/openonu-go-adapter/Voltha_ONUFlowChecks.robot
@@ -156,8 +156,8 @@
${result}= Prepare ONU Go Adapter ETCD Data For Json ${etcddata}
${jsondata}= To Json ${result}
${length}= Get Length ${jsondata}
- log ${jsondata}
- Should Not Be Empty ${jsondata}
+ Log ${jsondata}
+ Should Not Be Empty ${jsondata} Could not find ONU Go Adapter ETCD data
FOR ${INDEX} IN RANGE 0 ${length}
${value}= Get From List ${jsondata} ${INDEX}
${flowparams}= Get From Dictionary ${value['uni_config'][0]} flow_params
@@ -199,7 +199,7 @@
${jsondata}= To Json ${result}
${length}= Get Length ${jsondata}
log ${jsondata}
- Should Not Be Empty ${jsondata}
+ Should Not Be Empty ${jsondata} Could not find ONU Go Adapter ETCD data
FOR ${INDEX} IN RANGE 0 ${length}
${value}= Get From List ${jsondata} ${INDEX}
@{result_values}= Run Keyword And Ignore Error
diff --git a/tests/scale/Voltha_Scale_Tests.robot b/tests/scale/Voltha_Scale_Tests.robot
index b794b20..8d3d471 100755
--- a/tests/scale/Voltha_Scale_Tests.robot
+++ b/tests/scale/Voltha_Scale_Tests.robot
@@ -133,14 +133,14 @@
[Documentation] Check that all the flows has been stored in the logical device
[Tags] flow-before plot-voltha-flows-before
# NOTE fail the test immediately if we're trying to check flows without provisioning them
- Should Be Equal ${enableFlowProvisioning} true
+ Should Be Equal ${enableFlowProvisioning} true Flow provisioning is not enabled
Wait for Logical Devices flows ${workflow} ${total_onus} ${olt} false
... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP} ${timeout}
Flows validation in VOLTHA Adapters before subscriber provisioning
[Documentation] Check that all flows has been store in devices of type openolt
[Tags] flow-before plot-voltha-openolt-flows-before only-me
- Should Be Equal ${enableFlowProvisioning} true
+ Should Be Equal ${enableFlowProvisioning} true Flow provisioning is not enabled
Wait for OpenOLT Devices flows ${workflow} ${total_onus} ${olt} false
... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
@@ -148,7 +148,7 @@
[Documentation] Check that all the flows has been acknowledged
[Tags] flow-before plot-onos-flows-before
# NOTE fail the test immediately if we're trying to check flows without provisioning them
- Should Be Equal ${enableFlowProvisioning} true
+ Should Be Equal ${enableFlowProvisioning} true Flow provisioning is not enabled
${onos_devices}= Compute Device IDs
FOR ${deviceId} IN @{onos_devices}
@@ -234,7 +234,7 @@
Provision subscribers
[Documentation] Provision data plane flows for all the subscribers
[Tags] provision
- Should Be Equal ${enableSubscriberProvisioning} true
+ Should Be Equal ${enableSubscriberProvisioning} true Subscriber provisioning is not enabled
${onos_devices}= Compute Device IDs
FOR ${olt} IN @{onos_devices}
Provision all subscribers on device ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${ONOS_SSH_IP} ${ONOS_REST_PORT} ${olt}
@@ -244,7 +244,7 @@
[Documentation] Check that all the flows has been stored in the logical device
[Tags] flow-after plot-voltha-flows-after
# NOTE fail the test immediately if we're trying to check flows without provisioning them
- Should Be Equal ${enableFlowProvisioning} true
+ Should Be Equal ${enableFlowProvisioning} true Flow provisioning is not enabled
Wait for Logical Devices flows ${workflow} ${total_onus} ${olt} true
... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP} ${timeout}
@@ -252,7 +252,7 @@
Flows validation in VOLTHA Adapters after subscriber provisioning
[Documentation] Check that all flows has been store in devices of type openolt
[Tags] flow-after plot-voltha-openolt-flows-after only-me
- Should Be Equal ${enableFlowProvisioning} true
+ Should Be Equal ${enableFlowProvisioning} true Flow provisioning is not enabled
Wait for OpenOLT Devices flows ${workflow} ${total_onus} ${olt} true
... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
@@ -260,7 +260,7 @@
[Documentation] Check that all the flows has been acknowledged
[Tags] flow-after plot-onos-flows-after
# NOTE fail the test immediately if we're trying to check flows without provisioning them
- Should Be Equal ${enableFlowProvisioning} true
+ Should Be Equal ${enableFlowProvisioning} true Flow provisioning is not enabled
${onos_devices}= Compute Device IDs
FOR ${deviceId} IN @{onos_devices}
@@ -346,7 +346,7 @@
*** Keywords ***
Setup Suite
[Documentation] Setup test global variables, open an SSH connection to ONOS and starts a timer
- Set Suite Variable ${KUBECTL_CONFIG} export KUBECONFIG=%{KUBECONFIG}
+ Set Suite Variable ${KUBECTL_CONFIG} %{KUBECONFIG}
Set Suite Variable ${VOLTCTL_CONFIG} %{VOLTCONFIG}
${total_onus}= Evaluate ${olt} * ${pon} * ${onu}