src/test/cord-api/Tests/WorkflowValidations/ATT_Test001.robot

# Copyright 2017-present Open Networking Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

*** Settings ***
Documentation     Test various end-to-end scenarios with ATT workflow
Suite Setup       Setup Suite
Suite Teardown    Teardown Suite
Test Setup        Setup Test
Test Teardown     Teardown Test
Library           Collections
Library           String
Library           OperatingSystem
Library           XML
Library           RequestsLibrary
Library           ../../Framework/utils/utils.py
Resource          ../../Framework/utils/utils.robot
Library           ../../Framework/restApi.py
Resource          ../../Framework/Subscriber.robot
Resource          ../../Framework/ATTWorkFlowDriver.robot
Resource          ../../Framework/Kubernetes.robot
Resource          ../../Framework/ONU.robot
Resource          ../../Framework/OLT.robot
Resource          ../../Framework/DHCP.robot
Variables         ../../Properties/RestApiProperties.py

*** Variables ***
${POD_NAME}                 flex-pod1-olt
${KUBERNETES_CONFIGS_DIR}   ~/pod-configs/kubernetes-configs
${HELM_CHARTS_DIR}          ~/helm-charts
${WHITELIST_PATHFILE}       ${CURDIR}/data/${POD_NAME}/ATTWhiteList.json
${SUBSCRIBER_PATHFILE}      ${CURDIR}/data/${POD_NAME}/ATTSubscriber.json
${VOLT_DEVICE_PATHFILE}     ${CURDIR}/data/${POD_NAME}/RealOLTDevice.json
${KUBERNETES_CONF}          ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
${KUBERNETES_YAML}          ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.yml
${VOLTHA_POD_NUM}           8

*** Test Cases ***
ONU in Correct Location
    [Documentation]    Validates E2E Ping Connectivity and object states for the given scenario:
    ...    Configure whitelist with correct ONU location
    ...    Validate successful authentication/DHCP/E2E ping
    [Setup]    None
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    ACTIVE    ENABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    True    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    APPROVED
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    enabled
    Validate DHCP and Ping    True    True

ONU in Correct Location -> Remove ONU from Whitelist -> Add ONU to Whitelist
    [Documentation]    Validates E2E Ping Connectivity and object states for the given scenario:
    ...    Configure whitelist with correct ONU location
    ...    Validate successful authentication/DHCP/E2E ping
    ...    Remove ONU from whitelist
    ...    Validate failed authentication/DHCP/E2E ping
    ...    Add ONU to whitelist
    ...    Validate successful authentication/DHCP/E2E ping
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    ACTIVE    ENABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    True    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    APPROVED
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    enabled
    Validate DHCP and Ping    True    True
    Clean Up Linux
    Remove Whitelist
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    DISABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    False    wpa_supplicant.conf
    Validate DHCP and Ping    False    False
    Clean Up Linux
    Create Whitelist
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    True    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    APPROVED
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    enabled
    Validate DHCP and Ping    True    True

ONU in Correct Location -> ONU in Wrong Location -> ONU in Correct Location
    [Documentation]    Validates E2E Ping Connectivity and object states for the given scenario:
    ...    Configure whitelist with correct ONU location
    ...    Validate successful authentication/DHCP/E2E ping
    ...    Update whitelist with wrong ONU location
    ...    Validate failed authentication/DHCP/E2E ping
    ...    Update whitelist with correct ONU location
    ...    Validate successful authentication/DHCP/E2E ping
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    ACTIVE    ENABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    True    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    APPROVED
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    enabled
    Validate DHCP and Ping    True    True
    Clean Up Linux
    Update Whitelist with Wrong Location
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    DISABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    False    wpa_supplicant.conf
    Validate DHCP and Ping    False    False
    Clean Up Linux
    Update Whitelist with Correct Location
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    True    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    APPROVED
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    enabled
    Validate DHCP and Ping    True    True

ONU in Correct Location -> Remove Subscriber -> Create Subscriber
    [Documentation]    Validates E2E Ping Connectivity and object states for the given scenario:
    ...    Configure whitelist with correct ONU location
    ...    Validate successful authentication/DHCP/E2E ping
    ...    Remove subscriber model
    ...    Validate failed authentication/DHCP/E2E ping
    ...    Recreate subscriber model
    ...    Validate successful authentication/DHCP/E2E ping
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    ACTIVE    ENABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    True    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    APPROVED
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    enabled
    Validate DHCP and Ping    True    True
    Clean Up Linux
    Remove Subscriber
    Validate Authentication    False    wpa_supplicant.conf
    Validate DHCP and Ping    False    False
    Clean Up Linux
    Create Subscriber
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    True    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    APPROVED
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    enabled
    Validate DHCP and Ping    True    True

ONU in Correct Location (Skip Subscriber Provisioning) -> Provision Subscriber
    [Documentation]    Validates E2E Ping Connectivity and object states for the given scenario:
    ...    Configure whitelist with correct ONU location and skip provisioning subscriber
    ...    Validate failed authentication/DHCP/E2E ping
    ...    Provision subscriber
    ...    Validate successful authentication/DHCP/E2E ping
    [Setup]    None
    Create Whitelist
    Create VOLT
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    ACTIVE    ENABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Validate Authentication    False    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Validate DHCP and Ping    False    False
    Clean Up Linux
    Create Subscriber
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    True    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    APPROVED
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    enabled
    Validate DHCP and Ping    True    True

ONU in Correct Location (Skip Authentication)
    [Documentation]    Validates E2E Ping Connectivity and object states for the given scenario:
    ...    Configure whitelist with correct ONU location and skip RG authentication
    ...    Validate failed authentication/DHCP/E2E ping
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    ACTIVE    ENABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate DHCP and Ping    False    False

ONU not in Whitelist
    [Documentation]    Validates E2E Ping Connectivity and object states for the given scenario:
    ...    Skip whitelist configuration for ONU
    ...    Validate failed authentication/DHCP/E2E ping
    [Setup]    None
    Create Subscriber
    Create VOLT
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    UNKNOWN    DISABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    DISABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    False    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    DISABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate DHCP and Ping    False    False

ONU not in Whitelist (Skip Subscriber Provisioning) -> Add ONU to Whitelist -> Provision Subscriber
    [Documentation]    Validates E2E Ping Connectivity and object states for the given scenario:
    ...    Skip whitelist configuration for ONU and subscriber provisioning
    ...    Validate failed authentication/DHCP/E2E ping
    ...    Configure whitelist with correct ONU location
    ...    Validate failed authentication/DHCP/E2E ping
    ...    Provision subscriber
    ...    Validate successful authentication/DHCP/E2E ping
    [Setup]    None
    Create VOLT
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    UNKNOWN    DISABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    DISABLED    AWAITING
    Validate Authentication    False    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    DISABLED    AWAITING
    Validate DHCP and Ping    False    False
    Clean Up Linux
    Create Whitelist
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    ACTIVE    ENABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Validate Authentication    False    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Validate DHCP and Ping    False    False
    Clean Up Linux
    Create Subscriber
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    True    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    APPROVED
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    enabled
    Validate DHCP and Ping    True    True

ONU in Wrong Location
    [Documentation]    Validates E2E Ping Connectivity and object states for the given scenario:
    ...    Configure whitelist with wrong ONU location
    ...    Validate failed authentication/DHCP/E2E ping
    Update Whitelist with Wrong Location
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    UNKNOWN    DISABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    DISABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    False    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    DISABLED    AWAITING
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate DHCP and Ping    False    False

ONU with Wrong Location (Skip Subscriber Provisioning) -> ONU with Correct Location -> Provision Subscriber
    [Documentation]    Validates E2E Ping Connectivity and object states for the given scenario:
    ...    Configure whitelist with wrong ONU location and skip subscriber provisioning
    ...    Validate failed authentication/DHCP/E2E ping
    ...    Configure whitelist with correct ONU location
    ...    Validate failed authentication/DHCP/E2E ping
    ...    Provision subscriber
    ...    Validate successful authentication/DHCP/E2E ping
    [Setup]    None
    [Tags]    Debug
    Create VOLT
    Create Whitelist
    Update Whitelist with Wrong Location
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    UNKNOWN    DISABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    DISABLED    AWAITING
    Validate Authentication    False    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    DISABLED    AWAITING
    Validate DHCP and Ping    False    False
    Clean Up Linux
    Update Whitelist with Correct Location
    Wait Until Keyword Succeeds    300s    15s    Validate ONU States    ACTIVE    ENABLED
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Validate Authentication    False    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    AWAITING
    Validate DHCP and Ping    False    False
    Clean Up Linux
    Create Subscriber
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    awaiting-auth
    Validate Authentication    True    wpa_supplicant.conf
    Wait Until Keyword Succeeds    60s    2s    Validate ATT Workflow Driver SI    ENABLED    APPROVED
    Wait Until Keyword Succeeds    60s    2s    Validate Subscriber Status    enabled
    Validate DHCP and Ping    True    True

*** Keywords ***
Setup Suite
    ${auth} =    Create List    ${XOS_USER}    ${XOS_PASSWD}
    ${HEADERS}    Create Dictionary    Content-Type=application/json
    Create Session    ${server_ip}    http://${server_ip}:${server_port}    auth=${AUTH}    headers=${HEADERS}
    ${att_workflow_service_id}=    Get Service Owner Id    ${ATT_SERVICE}
    ${volt_service_id}=    Get Service Owner Id    ${VOLT_SERVICE}
    ${AttWhiteListList}=    utils.jsonToList    ${WHITELIST_PATHFILE}   AttWhiteListInfo
    Set Suite Variable    ${AttWhiteListList}
    ${AttWhiteListDict}=    utils.listToDict    ${AttWhiteListList}    0
    ${AttWhiteListDict}=    utils.setFieldValueInDict    ${AttWhiteListDict}    owner_id    ${att_workflow_service_id}
    ${onu_device}=   Get From Dictionary    ${AttWhiteListDict}    serial_number
    Set Global Variable    ${onu_device}
    ${onu_location}=   Get From Dictionary    ${AttWhiteListDict}    pon_port_id
    Set Global Variable    ${onu_location}
    ${SubscriberList}=    utils.jsonToList    ${SUBSCRIBER_PATHFILE}   SubscriberInfo
    Set Global Variable    ${SubscriberList}
    ${SubscriberDict}=    utils.listToDict    ${SubscriberList}    0
    ${s_tag}=    utils.getFieldValueFromDict    ${SubscriberDict}   s_tag
    ${c_tag}=    utils.getFieldValueFromDict    ${SubscriberDict}   c_tag
    ${VoltDeviceList}=    utils.jsonToList    ${VOLT_DEVICE_PATHFILE}   VOLTDeviceInfo
    Set Global Variable    ${VoltDeviceList}
    Set Suite Variable    ${s_tag}
    Set Suite Variable    ${c_tag}
    Set Global Variable    ${export_kubeconfig}    export KUBECONFIG=${KUBERNETES_CONF}
    # Read variables from yaml file
    ${src_ip}=    Evaluate    ${hosts}.get("src").get("ip")
    ${src_user}=    Evaluate    ${hosts}.get("src").get("user")
    ${src_pass}=    Evaluate    ${hosts}.get("src").get("pass")
    ${src_container_name}=    Evaluate    ${hosts}.get("src").get("container_name")
    ${src_iface}=    Evaluate    ${hosts}.get("src").get("dp_iface_name")
    ${dst_ip}=    Evaluate    ${hosts}.get("dst").get("ip")
    ${dst_user} =    Evaluate    ${hosts}.get("dst").get("user")
    ${dst_pass}=    Evaluate    ${hosts}.get("dst").get("pass")
    ${dst_container_name}=    Evaluate    ${hosts}.get("dst").get("container_name")
    ${dst_dp_iface}=    Evaluate    ${hosts}.get("dst").get("dp_iface_name")
    ${dst_dp_ip}=    Evaluate    ${hosts}.get("dst").get("dp_iface_ip_qinq")
    ${olt_ip}=    Evaluate    ${olts}[0].get("ip")
    ${olt_user}=    Evaluate    ${olts}[0].get("user")
    ${olt_pass}=    Evaluate    ${olts}[0].get("pass")
    Set Suite Variable    ${src_ip}
    Set Suite Variable    ${src_user}
    Set Suite Variable    ${src_pass}
    Set Suite Variable    ${src_container_name}
    Set Suite Variable    ${src_iface}
    Set Suite Variable    ${dst_ip}
    Set Suite Variable    ${dst_user}
    Set Suite Variable    ${dst_pass}
    Set Suite Variable    ${dst_container_name}
    Set Suite Variable    ${dst_dp_iface}
    Set Suite Variable    ${dst_dp_ip}
    Set Suite Variable    ${olt_ip}
    Set Suite Variable    ${olt_user}
    Set Suite Variable    ${olt_pass}

Teardown Suite
    [Documentation]    Performs any additional cleanup required
    Log    Suite Teardown cleanup
    Delete All Sessions

Setup Test
    [Documentation]    Re-create Subscriber, whitelist, and olt-device models to test
    Log    Re-creating objects
    Create Whitelist
    Create Subscriber
    Create VOLT

Create Whitelist
    ${AttWhiteListDict}=    utils.listToDict    ${AttWhiteListList}    0
    CORD Post    ${ATT_WHITELIST}    ${AttWhiteListDict}

Create Subscriber
    ${SubscriberDict}=    utils.listToDict    ${SubscriberList}    0
    Wait Until Keyword Succeeds    120s    15s    CORD Post    ${VOLT_SUBSCRIBER}    ${SubscriberDict}

Create VOLT
    ${VoltDeviceDict}=    utils.listToDict    ${VoltDeviceList}    0
    CORD Post    ${VOLT_DEVICE}    ${VoltDeviceDict}

Teardown Test
    [Documentation]    Delete xos objects, kills processes and cleans up interfaces on src+dst servers
    Clean Up Linux
    Clean Up XOS

Clean Up Linux
    [Documentation]    Kill processes and clean up interfaces on src+dst servers
    Run Keyword And Ignore Error    Kill Linux Process    [w]pa_supplicant    ${src_ip}    ${src_user}    ${src_pass}    ${src_container_name}
    Run Keyword And Ignore Error    Kill Linux Process    [d]hclient    ${src_ip}    ${src_user}    ${src_pass}    ${src_container_name}
    Run Keyword If    '${dst_ip}' != '${None}'    Run Keyword And Ignore Error    Kill Linux Process    [d]hcpd    ${dst_ip}    ${dst_user}    ${dst_pass}    ${dst_container_name}
    Delete IP Addresses from Interface on Remote Host    ${src_iface}    ${src_ip}    ${src_user}    ${src_pass}    ${src_container_name}
    Run Keyword If    '${dst_ip}' != '${None}'    Delete Interface on Remote Host    ${dst_dp_iface}.${s_tag}    ${dst_ip}    ${dst_user}    ${dst_pass}    ${dst_container_name}

Clean Up XOS
    [Documentation]    Clean up all XOS objects and reinstall voltha after OLT reboots
    Wait Until Keyword Succeeds    60s    2s    Clean Up Objects    ${VOLT_SUBSCRIBER}
    Wait Until Keyword Succeeds    60s    2s    Clean Up Objects    ${VOLT_DEVICE}
    Wait Until Keyword Succeeds    60s    2s    Clean Up Objects    ${ATT_WHITELIST}
    Wait Until Keyword Succeeds    120s    10s    Check Remote System Reachability    False    ${olt_ip}
    Wait Until Keyword Succeeds    120s    10s    Check Remote System Reachability    True    ${olt_ip}
    Wait Until Keyword Succeeds    120s    10s    Openolt is Up    ${olt_ip}    ${olt_user}    ${olt_pass}
    Wait Until Keyword Succeeds    300s    10s    Reinstall Voltha

Remove Whitelist
    ${whitelist_id}=    Retrieve Whitelist Entry    ${onu_device}
    CORD Delete    ${ATT_WHITELIST}    ${whitelist_id}

Update Whitelist with Wrong Location
    ${whitelist_id}=    Retrieve Whitelist Entry    ${onu_device}
    CORD Put    ${ATT_WHITELIST}    {"pon_port_id": 55 }    ${whitelist_id}

Update Whitelist with Correct Location
    ${whitelist_id}=    Retrieve Whitelist Entry    ${onu_device}
    CORD Put    ${ATT_WHITELIST}    {"pon_port_id": ${onu_location} }    ${whitelist_id}

Validate ONU States
    [Arguments]    ${expected_op_status}    ${expected_admin_status}
    ${operational_status}    ${admin_status}    ONU Status Check    ${onu_device}
    Should Be Equal    ${operational_status}    ${expected_op_status}
    Should Be Equal    ${admin_status}    ${expected_admin_status}

Validate ATT Workflow Driver SI
    [Arguments]    ${expected_status}    ${expected_auth_status}
    ${onu_state}   ${authentication_status}   Service Instance Status Check    ${onu_device}
    Should Be Equal    ${onu_state}    ${expected_status}
    Should Be Equal    ${authentication_status}    ${expected_auth_status}

Validate Subscriber Status
    [Arguments]    ${exepected_status}
    ${status}    Subscriber Status Check    ${onu_device}
    Should Be Equal    ${status}    ${exepected_status}

Validate Authentication
    [Arguments]    ${auth_pass}    ${conf_file}
    [Documentation]    Executes a particular auth request on the RG and verifies if it succeeds. auth_pass determines if authentication should pass
    Send EAPOL Message    ${src_iface}    ${conf_file}    ${src_ip}    ${src_user}    ${src_pass}    ${src_container_name}
    Run Keyword If    '${auth_pass}' == 'True'    Wait Until Keyword Succeeds    30s    2s    Check Remote File Contents    True    /tmp/wpa.log    authentication completed successfully    ${src_ip}    ${src_user}    ${src_pass}    ${src_container_name}
    Run Keyword If    '${auth_pass}' == 'False'    Sleep    10s
    Run Keyword If    '${auth_pass}' == 'False'    Check Remote File Contents    False    /tmp/wpa.log    authentication completed successfully    ${src_ip}    ${src_user}    ${src_pass}    ${src_container_name}

Validate DHCP and Ping
    [Arguments]    ${dhcp_should_pass}    ${ping_should_pass}
    Run Keyword If    '${dst_ip}' != '${None}'    Run Keywords
    ...    Add Double Vlan Interface on Host    ${dst_dp_iface}    ${s_tag}    ${c_tag}    ${dst_ip}    ${dst_user}    ${dst_pass}    ${dst_container_name}    AND
    ...    Add IP Address on Interface on Host    ${dst_dp_ip}/24    ${dst_dp_iface}.${s_tag}.${c_tag}    ${dst_ip}    ${dst_user}    ${dst_pass}    ${dst_container_name}    AND
    ...    Start DHCP Server on Remote Host    ${dst_dp_iface}.${s_tag}.${c_tag}    ${dst_ip}    ${dst_user}    ${dst_pass}    ${dst_container_name}
    Send Dhclient Request    ${src_iface}    ${src_ip}    ${src_user}    ${src_pass}    ${src_container_name}
    Run Keyword If    '${dhcp_should_pass}' == 'True'    Wait Until Keyword Succeeds    60s    5s    Check IPv4 Address on DHCP Client    True    ${src_iface}    ${src_ip}    ${src_user}    ${src_pass}    ${src_container_name}
    Run Keyword If    '${dhcp_should_pass}' == 'False'    Sleep    10s
    Run Keyword If    '${dhcp_should_pass}' == 'False'    Check IPv4 Address on DHCP Client    False    ${src_iface}    ${src_ip}    ${src_user}    ${src_pass}    ${src_container_name}
    Run Keyword If    '${ping_should_pass}' == 'True'    Wait Until Keyword Succeeds    60s    5s    Check Ping    True    ${dst_dp_ip}    ${src_iface}    ${src_ip}    ${src_user}    ${src_pass}    ${src_container_name}
    ...                                          ELSE    Wait Until Keyword Succeeds    60s    5s    Check Ping    False    ${dst_dp_ip}    ${src_iface}    ${src_ip}    ${src_user}    ${src_pass}    ${src_container_name}

Reinstall Voltha
    Run    ${export_kubeconfig}; helm delete --purge voltha
    Wait Until Keyword Succeeds    60s    10s    Helm Chart is Removed    voltha
    Wait Until Keyword Succeeds    120s    10s    Kubernetes PODs in Namespace are Removed    voltha
    Run    ${export_kubeconfig}; cd ${HELM_CHARTS_DIR}; helm repo add incubator https://kubernetes-charts-incubator.storage.googleapis.com/
    Run    ${export_kubeconfig}; cd ${HELM_CHARTS_DIR}; helm dep up voltha
    Run    ${export_kubeconfig}; helm install -n voltha -f ${KUBERNETES_YAML} --set etcd-operator.customResources.createEtcdClusterCRD=false ${HELM_CHARTS_DIR}/voltha
    Run    ${export_kubeconfig}; helm upgrade -f ${KUBERNETES_YAML} --set etcd-operator.customResources.createEtcdClusterCRD=true voltha ${HELM_CHARTS_DIR}/voltha
    Wait Until Keyword Succeeds    60s    10s    Kubernetes PODs in Namespace are Running    voltha    ${VOLTHA_POD_NUM}
    Sleep    10s