blob: e34646fc95797d149538f6c1fe0fd48392756cbe [file] [log] [blame]
# Copyright 2017-present Open Networking Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
*** Settings ***
Documentation Test various end-to-end scenarios with ATT workflow
Suite Setup Setup Suite
Suite Teardown Teardown Suite
Test Setup Setup Test
Test Teardown Teardown Test
Library Collections
Library String
Library OperatingSystem
Library XML
Library RequestsLibrary
Library ../../Framework/utils/utils.py
Resource ../../Framework/utils/utils.robot
Library ../../Framework/restApi.py
Resource ../../Framework/Subscriber.robot
Resource ../../Framework/ATTWorkFlowDriver.robot
Resource ../../Framework/Kubernetes.robot
Resource ../../Framework/ONU.robot
Resource ../../Framework/OLT.robot
Resource ../../Framework/DHCP.robot
Variables ../../Properties/RestApiProperties.py
*** Variables ***
${POD_NAME} flex-pod1-olt
${KUBERNETES_CONFIGS_DIR} ~/pod-configs/kubernetes-configs
${HELM_CHARTS_DIR} ~/helm-charts
${WHITELIST_PATHFILE} ${CURDIR}/data/${POD_NAME}/ATTWhiteList.json
${SUBSCRIBER_PATHFILE} ${CURDIR}/data/${POD_NAME}/ATTSubscriber.json
${VOLT_DEVICE_PATHFILE} ${CURDIR}/data/${POD_NAME}/RealOLTDevice.json
${KUBERNETES_CONF} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
${KUBERNETES_YAML} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.yml
${VOLTHA_POD_NUM} 8
*** Test Cases ***
ONU in Correct Location
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
... Configure whitelist with correct ONU location
... Validate successful authentication/DHCP/E2E ping
[Setup] None
Wait Until Keyword Succeeds 300s 15s Validate ONU States ACTIVE ENABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication True wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED APPROVED
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status enabled
Validate DHCP and Ping True True
ONU in Correct Location -> Remove ONU from Whitelist -> Add ONU to Whitelist
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
... Configure whitelist with correct ONU location
... Validate successful authentication/DHCP/E2E ping
... Remove ONU from whitelist
... Validate failed authentication/DHCP/E2E ping
... Add ONU to whitelist
... Validate successful authentication/DHCP/E2E ping
Wait Until Keyword Succeeds 300s 15s Validate ONU States ACTIVE ENABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication True wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED APPROVED
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status enabled
Validate DHCP and Ping True True
Clean Up Linux
Remove Whitelist
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI DISABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication False wpa_supplicant.conf
Validate DHCP and Ping False False
Clean Up Linux
Create Whitelist
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication True wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED APPROVED
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status enabled
Validate DHCP and Ping True True
ONU in Correct Location -> ONU in Wrong Location -> ONU in Correct Location
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
... Configure whitelist with correct ONU location
... Validate successful authentication/DHCP/E2E ping
... Update whitelist with wrong ONU location
... Validate failed authentication/DHCP/E2E ping
... Update whitelist with correct ONU location
... Validate successful authentication/DHCP/E2E ping
Wait Until Keyword Succeeds 300s 15s Validate ONU States ACTIVE ENABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication True wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED APPROVED
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status enabled
Validate DHCP and Ping True True
Clean Up Linux
Update Whitelist with Wrong Location
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI DISABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication False wpa_supplicant.conf
Validate DHCP and Ping False False
Clean Up Linux
Update Whitelist with Correct Location
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication True wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED APPROVED
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status enabled
Validate DHCP and Ping True True
ONU in Correct Location -> Remove Subscriber -> Create Subscriber
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
... Configure whitelist with correct ONU location
... Validate successful authentication/DHCP/E2E ping
... Remove subscriber model
... Validate failed authentication/DHCP/E2E ping
... Recreate subscriber model
... Validate successful authentication/DHCP/E2E ping
Wait Until Keyword Succeeds 300s 15s Validate ONU States ACTIVE ENABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication True wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED APPROVED
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status enabled
Validate DHCP and Ping True True
Clean Up Linux
Remove Subscriber
Validate Authentication False wpa_supplicant.conf
Validate DHCP and Ping False False
Clean Up Linux
Create Subscriber
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication True wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED APPROVED
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status enabled
Validate DHCP and Ping True True
ONU in Correct Location (Skip Subscriber Provisioning) -> Provision Subscriber
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
... Configure whitelist with correct ONU location and skip provisioning subscriber
... Validate failed authentication/DHCP/E2E ping
... Provision subscriber
... Validate successful authentication/DHCP/E2E ping
[Setup] None
Create Whitelist
Create VOLT
Wait Until Keyword Succeeds 300s 15s Validate ONU States ACTIVE ENABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Validate Authentication False wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Validate DHCP and Ping False False
Clean Up Linux
Create Subscriber
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication True wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED APPROVED
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status enabled
Validate DHCP and Ping True True
ONU in Correct Location (Skip Authentication)
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
... Configure whitelist with correct ONU location and skip RG authentication
... Validate failed authentication/DHCP/E2E ping
Wait Until Keyword Succeeds 300s 15s Validate ONU States ACTIVE ENABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate DHCP and Ping False False
ONU not in Whitelist
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
... Skip whitelist configuration for ONU
... Validate failed authentication/DHCP/E2E ping
[Setup] None
Create Subscriber
Create VOLT
Wait Until Keyword Succeeds 300s 15s Validate ONU States UNKNOWN DISABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI DISABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication False wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI DISABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate DHCP and Ping False False
ONU not in Whitelist (Skip Subscriber Provisioning) -> Add ONU to Whitelist -> Provision Subscriber
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
... Skip whitelist configuration for ONU and subscriber provisioning
... Validate failed authentication/DHCP/E2E ping
... Configure whitelist with correct ONU location
... Validate failed authentication/DHCP/E2E ping
... Provision subscriber
... Validate successful authentication/DHCP/E2E ping
[Setup] None
Create VOLT
Wait Until Keyword Succeeds 300s 15s Validate ONU States UNKNOWN DISABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI DISABLED AWAITING
Validate Authentication False wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI DISABLED AWAITING
Validate DHCP and Ping False False
Clean Up Linux
Create Whitelist
Wait Until Keyword Succeeds 300s 15s Validate ONU States ACTIVE ENABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Validate Authentication False wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Validate DHCP and Ping False False
Clean Up Linux
Create Subscriber
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication True wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED APPROVED
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status enabled
Validate DHCP and Ping True True
ONU in Wrong Location
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
... Configure whitelist with wrong ONU location
... Validate failed authentication/DHCP/E2E ping
Update Whitelist with Wrong Location
Wait Until Keyword Succeeds 300s 15s Validate ONU States UNKNOWN DISABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI DISABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication False wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI DISABLED AWAITING
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate DHCP and Ping False False
ONU with Wrong Location (Skip Subscriber Provisioning) -> ONU with Correct Location -> Provision Subscriber
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
... Configure whitelist with wrong ONU location and skip subscriber provisioning
... Validate failed authentication/DHCP/E2E ping
... Configure whitelist with correct ONU location
... Validate failed authentication/DHCP/E2E ping
... Provision subscriber
... Validate successful authentication/DHCP/E2E ping
[Setup] None
[Tags] Debug
Create VOLT
Create Whitelist
Update Whitelist with Wrong Location
Wait Until Keyword Succeeds 300s 15s Validate ONU States UNKNOWN DISABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI DISABLED AWAITING
Validate Authentication False wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI DISABLED AWAITING
Validate DHCP and Ping False False
Clean Up Linux
Update Whitelist with Correct Location
Wait Until Keyword Succeeds 300s 15s Validate ONU States ACTIVE ENABLED
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Validate Authentication False wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED AWAITING
Validate DHCP and Ping False False
Clean Up Linux
Create Subscriber
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status awaiting-auth
Validate Authentication True wpa_supplicant.conf
Wait Until Keyword Succeeds 60s 2s Validate ATT Workflow Driver SI ENABLED APPROVED
Wait Until Keyword Succeeds 60s 2s Validate Subscriber Status enabled
Validate DHCP and Ping True True
*** Keywords ***
Setup Suite
${auth} = Create List ${XOS_USER} ${XOS_PASSWD}
${HEADERS} Create Dictionary Content-Type=application/json
Create Session ${server_ip} http://${server_ip}:${server_port} auth=${AUTH} headers=${HEADERS}
${att_workflow_service_id}= Get Service Owner Id ${ATT_SERVICE}
${volt_service_id}= Get Service Owner Id ${VOLT_SERVICE}
${AttWhiteListList}= utils.jsonToList ${WHITELIST_PATHFILE} AttWhiteListInfo
Set Suite Variable ${AttWhiteListList}
${AttWhiteListDict}= utils.listToDict ${AttWhiteListList} 0
${AttWhiteListDict}= utils.setFieldValueInDict ${AttWhiteListDict} owner_id ${att_workflow_service_id}
${onu_device}= Get From Dictionary ${AttWhiteListDict} serial_number
Set Global Variable ${onu_device}
${onu_location}= Get From Dictionary ${AttWhiteListDict} pon_port_id
Set Global Variable ${onu_location}
${SubscriberList}= utils.jsonToList ${SUBSCRIBER_PATHFILE} SubscriberInfo
Set Global Variable ${SubscriberList}
${SubscriberDict}= utils.listToDict ${SubscriberList} 0
${s_tag}= utils.getFieldValueFromDict ${SubscriberDict} s_tag
${c_tag}= utils.getFieldValueFromDict ${SubscriberDict} c_tag
${VoltDeviceList}= utils.jsonToList ${VOLT_DEVICE_PATHFILE} VOLTDeviceInfo
Set Global Variable ${VoltDeviceList}
Set Suite Variable ${s_tag}
Set Suite Variable ${c_tag}
Set Global Variable ${export_kubeconfig} export KUBECONFIG=${KUBERNETES_CONF}
# Read variables from yaml file
${src_ip}= Evaluate ${hosts}.get("src").get("ip")
${src_user}= Evaluate ${hosts}.get("src").get("user")
${src_pass}= Evaluate ${hosts}.get("src").get("pass")
${src_container_name}= Evaluate ${hosts}.get("src").get("container_name")
${src_iface}= Evaluate ${hosts}.get("src").get("dp_iface_name")
${dst_ip}= Evaluate ${hosts}.get("dst").get("ip")
${dst_user} = Evaluate ${hosts}.get("dst").get("user")
${dst_pass}= Evaluate ${hosts}.get("dst").get("pass")
${dst_container_name}= Evaluate ${hosts}.get("dst").get("container_name")
${dst_dp_iface}= Evaluate ${hosts}.get("dst").get("dp_iface_name")
${dst_dp_ip}= Evaluate ${hosts}.get("dst").get("dp_iface_ip_qinq")
${olt_ip}= Evaluate ${olts}[0].get("ip")
${olt_user}= Evaluate ${olts}[0].get("user")
${olt_pass}= Evaluate ${olts}[0].get("pass")
Set Suite Variable ${src_ip}
Set Suite Variable ${src_user}
Set Suite Variable ${src_pass}
Set Suite Variable ${src_container_name}
Set Suite Variable ${src_iface}
Set Suite Variable ${dst_ip}
Set Suite Variable ${dst_user}
Set Suite Variable ${dst_pass}
Set Suite Variable ${dst_container_name}
Set Suite Variable ${dst_dp_iface}
Set Suite Variable ${dst_dp_ip}
Set Suite Variable ${olt_ip}
Set Suite Variable ${olt_user}
Set Suite Variable ${olt_pass}
Teardown Suite
[Documentation] Performs any additional cleanup required
Log Suite Teardown cleanup
Delete All Sessions
Setup Test
[Documentation] Re-create Subscriber, whitelist, and olt-device models to test
Log Re-creating objects
Create Whitelist
Create Subscriber
Create VOLT
Create Whitelist
${AttWhiteListDict}= utils.listToDict ${AttWhiteListList} 0
CORD Post ${ATT_WHITELIST} ${AttWhiteListDict}
Create Subscriber
${SubscriberDict}= utils.listToDict ${SubscriberList} 0
Wait Until Keyword Succeeds 120s 15s CORD Post ${VOLT_SUBSCRIBER} ${SubscriberDict}
Create VOLT
${VoltDeviceDict}= utils.listToDict ${VoltDeviceList} 0
CORD Post ${VOLT_DEVICE} ${VoltDeviceDict}
Teardown Test
[Documentation] Delete xos objects, kills processes and cleans up interfaces on src+dst servers
Clean Up Linux
Clean Up XOS
Clean Up Linux
[Documentation] Kill processes and clean up interfaces on src+dst servers
Run Keyword And Ignore Error Kill Linux Process [w]pa_supplicant ${src_ip} ${src_user} ${src_pass} ${src_container_name}
Run Keyword And Ignore Error Kill Linux Process [d]hclient ${src_ip} ${src_user} ${src_pass} ${src_container_name}
Run Keyword If '${dst_ip}' != '${None}' Run Keyword And Ignore Error Kill Linux Process [d]hcpd ${dst_ip} ${dst_user} ${dst_pass} ${dst_container_name}
Delete IP Addresses from Interface on Remote Host ${src_iface} ${src_ip} ${src_user} ${src_pass} ${src_container_name}
Run Keyword If '${dst_ip}' != '${None}' Delete Interface on Remote Host ${dst_dp_iface}.${s_tag} ${dst_ip} ${dst_user} ${dst_pass} ${dst_container_name}
Clean Up XOS
[Documentation] Clean up all XOS objects and reinstall voltha after OLT reboots
Wait Until Keyword Succeeds 60s 2s Clean Up Objects ${VOLT_SUBSCRIBER}
Wait Until Keyword Succeeds 60s 2s Clean Up Objects ${VOLT_DEVICE}
Wait Until Keyword Succeeds 60s 2s Clean Up Objects ${ATT_WHITELIST}
Wait Until Keyword Succeeds 120s 10s Check Remote System Reachability False ${olt_ip}
Wait Until Keyword Succeeds 120s 10s Check Remote System Reachability True ${olt_ip}
Wait Until Keyword Succeeds 120s 10s Openolt is Up ${olt_ip} ${olt_user} ${olt_pass}
Wait Until Keyword Succeeds 300s 10s Reinstall Voltha
Remove Whitelist
${whitelist_id}= Retrieve Whitelist Entry ${onu_device}
CORD Delete ${ATT_WHITELIST} ${whitelist_id}
Update Whitelist with Wrong Location
${whitelist_id}= Retrieve Whitelist Entry ${onu_device}
CORD Put ${ATT_WHITELIST} {"pon_port_id": 55 } ${whitelist_id}
Update Whitelist with Correct Location
${whitelist_id}= Retrieve Whitelist Entry ${onu_device}
CORD Put ${ATT_WHITELIST} {"pon_port_id": ${onu_location} } ${whitelist_id}
Validate ONU States
[Arguments] ${expected_op_status} ${expected_admin_status}
${operational_status} ${admin_status} ONU Status Check ${onu_device}
Should Be Equal ${operational_status} ${expected_op_status}
Should Be Equal ${admin_status} ${expected_admin_status}
Validate ATT Workflow Driver SI
[Arguments] ${expected_status} ${expected_auth_status}
${onu_state} ${authentication_status} Service Instance Status Check ${onu_device}
Should Be Equal ${onu_state} ${expected_status}
Should Be Equal ${authentication_status} ${expected_auth_status}
Validate Subscriber Status
[Arguments] ${exepected_status}
${status} Subscriber Status Check ${onu_device}
Should Be Equal ${status} ${exepected_status}
Validate Authentication
[Arguments] ${auth_pass} ${conf_file}
[Documentation] Executes a particular auth request on the RG and verifies if it succeeds. auth_pass determines if authentication should pass
Send EAPOL Message ${src_iface} ${conf_file} ${src_ip} ${src_user} ${src_pass} ${src_container_name}
Run Keyword If '${auth_pass}' == 'True' Wait Until Keyword Succeeds 30s 2s Check Remote File Contents True /tmp/wpa.log authentication completed successfully ${src_ip} ${src_user} ${src_pass} ${src_container_name}
Run Keyword If '${auth_pass}' == 'False' Sleep 10s
Run Keyword If '${auth_pass}' == 'False' Check Remote File Contents False /tmp/wpa.log authentication completed successfully ${src_ip} ${src_user} ${src_pass} ${src_container_name}
Validate DHCP and Ping
[Arguments] ${dhcp_should_pass} ${ping_should_pass}
Run Keyword If '${dst_ip}' != '${None}' Run Keywords
... Add Double Vlan Interface on Host ${dst_dp_iface} ${s_tag} ${c_tag} ${dst_ip} ${dst_user} ${dst_pass} ${dst_container_name} AND
... Add IP Address on Interface on Host ${dst_dp_ip}/24 ${dst_dp_iface}.${s_tag}.${c_tag} ${dst_ip} ${dst_user} ${dst_pass} ${dst_container_name} AND
... Start DHCP Server on Remote Host ${dst_dp_iface}.${s_tag}.${c_tag} ${dst_ip} ${dst_user} ${dst_pass} ${dst_container_name}
Send Dhclient Request ${src_iface} ${src_ip} ${src_user} ${src_pass} ${src_container_name}
Run Keyword If '${dhcp_should_pass}' == 'True' Wait Until Keyword Succeeds 60s 5s Check IPv4 Address on DHCP Client True ${src_iface} ${src_ip} ${src_user} ${src_pass} ${src_container_name}
Run Keyword If '${dhcp_should_pass}' == 'False' Sleep 10s
Run Keyword If '${dhcp_should_pass}' == 'False' Check IPv4 Address on DHCP Client False ${src_iface} ${src_ip} ${src_user} ${src_pass} ${src_container_name}
Run Keyword If '${ping_should_pass}' == 'True' Wait Until Keyword Succeeds 60s 5s Check Ping True ${dst_dp_ip} ${src_iface} ${src_ip} ${src_user} ${src_pass} ${src_container_name}
... ELSE Wait Until Keyword Succeeds 60s 5s Check Ping False ${dst_dp_ip} ${src_iface} ${src_ip} ${src_user} ${src_pass} ${src_container_name}
Reinstall Voltha
Run ${export_kubeconfig}; helm delete --purge voltha
Wait Until Keyword Succeeds 60s 10s Helm Chart is Removed voltha
Wait Until Keyword Succeeds 120s 10s Kubernetes PODs in Namespace are Removed voltha
Run ${export_kubeconfig}; cd ${HELM_CHARTS_DIR}; helm repo add incubator https://kubernetes-charts-incubator.storage.googleapis.com/
Run ${export_kubeconfig}; cd ${HELM_CHARTS_DIR}; helm dep up voltha
Run ${export_kubeconfig}; helm install -n voltha -f ${KUBERNETES_YAML} --set etcd-operator.customResources.createEtcdClusterCRD=false ${HELM_CHARTS_DIR}/voltha
Run ${export_kubeconfig}; helm upgrade -f ${KUBERNETES_YAML} --set etcd-operator.customResources.createEtcdClusterCRD=true voltha ${HELM_CHARTS_DIR}/voltha
Wait Until Keyword Succeeds 60s 10s Kubernetes PODs in Namespace are Running voltha ${VOLTHA_POD_NUM}
Sleep 10s