blob: f4af894dd8bd8a6f159cdb4539eadd097b811892 [file] [log] [blame]
# Copyright 2021-2024 Open Networking Foundation (ONF) and the ONF Contributors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# FIXME Can we use the same test against BBSim and Hardware?
*** Settings ***
Documentation Test Voltha Components Software Upgrade
Suite Setup Setup Suite
Test Setup Setup
Test Teardown Teardown
Suite Teardown Teardown Suite
Library Collections
Library String
Library OperatingSystem
Library XML
Library RequestsLibrary
Library ../../libraries/DependencyLibrary.py
Resource ../../libraries/onos.robot
Resource ../../libraries/voltctl.robot
Resource ../../libraries/voltha.robot
Resource ../../libraries/utils.robot
Resource ../../libraries/k8s.robot
Resource ../../variables/variables.robot
Resource ../../libraries/power_switch.robot
*** Variables ***
${POD_NAME} flex-ocp-cord
${KUBERNETES_CONF} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
${KUBERNETES_CONFIGS_DIR} ~/pod-configs/kubernetes-configs
#${KUBERNETES_CONFIGS_DIR} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
${KUBERNETES_YAML} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.yml
${HELM_CHARTS_DIR} ~/helm-charts
${VOLTHA_POD_NUM} 8
${NAMESPACE} voltha
${INFRA_NAMESPACE} default
# For below variable value, using deployment name as using grep for
# parsing radius pod name, we can also use full radius pod name
${RESTART_POD_NAME} radius
${timeout} 60s
${of_id} 0
${logical_id} 0
${has_dataplane} True
${teardown_device} False
${scripts} ../../scripts
# Per-test logging on failure is turned off by default; set this variable to enable
${container_log_dir} ${None}
# logging flag to enable Collect Logs, can be passed via the command line too
# example: -v logging:False
${logging} True
${suppressaddsubscriber} True
# Voltha Components to Test for Software Upgrade need to be passed in the following variable in format:
# <comp-label>,<comp-container>,<comp-image>*<comp-label>,<comp-container>,<comp-image>*
# Example: adapter-open-olt,adapter-open-olt,voltha/voltha-openolt-adapter:3.1.3*
${voltha_comps_under_test} ${EMPTY}
*** Test Cases ***
Test Voltha Components Minor Version Upgrade
[Documentation] Validates the Voltha Components Minor Version Upgrade doesn't affect the system functionality
... Performs the sanity and verifies all the ONUs are authenticated/DHCP/pingable
... Requirement: Components to test needs to be passed in robot command variable 'voltha_comps_under_test' in the format:
... <comp-label>,<comp-container>,<comp-image>*<comp-label>,<comp-container>,<comp-image>*
... Check [VOL-3843] for more details
[Tags] functional VolthaCompMinorVerUpgrade
[Setup] Start Logging VolthaCompMinorVerUpgrade
[Teardown] Run Keywords Collect Logs
... AND Stop Logging VolthaCompMinorVerUpgrade
... AND Delete All Devices and Verify
# Add OLT device
Setup
# Performing Sanity Test to make sure subscribers are all DHCP and pingable
Run Keyword If ${has_dataplane} Clean Up Linux
Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
Log ${podStatusOutput}
${countBeforeUpgrade}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
${num_comps_under_test}= Get Length ${list_voltha_comps_under_test}
FOR ${I} IN RANGE 0 ${num_comps_under_test}
${label}= Set Variable ${list_voltha_comps_under_test}[${I}][label]
${container}= Set Variable ${list_voltha_comps_under_test}[${I}][container]
${image}= Set Variable ${list_voltha_comps_under_test}[${I}][image]
${pod_image} ${app_ver} ${helm_chart} Get Pod Image And App Version And Helm Chart By Label
... ${NAMESPACE} app ${label}
Log ${label}: image, app ver & helm chart before upgrade: ${pod_image}, ${app_ver} & ${helm_chart}
${deployment}= Wait Until Keyword Succeeds ${timeout} 15s
... Get K8s Deployment by Pod Label ${NAMESPACE} app ${label}
Wait Until Keyword Succeeds ${timeout} 15s Deploy Pod New Image ${NAMESPACE} ${deployment}
... ${container} ${image}
Wait Until Keyword Succeeds ${timeout} 3s Validate Pods Status By Label ${NAMESPACE}
... app ${label} Running
Wait Until Keyword Succeeds ${timeout} 3s Pods Are Ready By Label ${NAMESPACE} app ${label}
Wait Until Keyword Succeeds ${timeout} 3s Verify Pod Image ${NAMESPACE} app ${label} ${image}
${pod_image_1} ${app_ver_1} ${helm_chart_1} Get Pod Image And App Version And Helm Chart By Label
... ${NAMESPACE} app ${label}
Log ${label}: image, app ver & helm chart after upgrade: ${pod_image_1}, ${app_ver_1} & ${helm_chart_1}
Restart VOLTHA Port Forward voltha-api
# Static sleep to let voltctl tcp connection establish
Sleep 5s
Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test ${suppressaddsubscriber}
END
${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
Log ${podStatusOutput}
${countAfterUpgrade}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
Should Be Equal As Strings ${countAfterUpgrade} ${countBeforeUpgrade}
# Additional Verification
Wait Until Keyword Succeeds ${timeout} 2s Delete All Devices and Verify
Setup
Run Keyword If ${has_dataplane} Clean Up Linux
Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
Test Voltha Components Minor Version Rolling Upgrade
[Documentation] Validates that the system can handle operations during software upgrade for minor versions.
... Requirement: Components to test needs to be passed in robot command variable 'voltha_comps_under_test' in the format:
... <comp-label>,<comp-container>,<comp-image>*<comp-label>,<comp-container>,<comp-image>*
... Check [VOL-4534] for more details
[Tags] functional VolthaCompMinorVerRollingUpgrade
[Setup] Start Logging VolthaCompMinorVerRollingUpgrade
[Teardown] Run Keywords Collect Logs
... AND Stop Logging VolthaCompMinorVerRollingUpgrade
... AND Delete All Devices and Verify
# Add OLT device
Setup
Run Keyword If ${has_dataplane} Clean Up Linux
FOR ${I} IN RANGE 0 ${olt_count}
${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
... ${olt_serial_number}
Wait Until Keyword Succeeds ${timeout} 2s Device Is Available In ONOS
... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT} ${of_id}
END
FOR ${I} IN RANGE 0 ${num_all_onus}
${src}= Set Variable ${hosts.src[${I}]}
${dst}= Set Variable ${hosts.dst[${I}]}
${of_id}= Get ofID From OLT List ${src['olt']}
${onu_device_id}= Get Device ID From SN ${src['onu']}
${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
... ${of_id} ${src['uni_id']}
# Check ONU port is Enabled in ONOS
Wait Until Keyword Succeeds 120s 2s
... Verify UNI Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']} ${src['uni_id']}
Wait Until Keyword Succeeds ${timeout} 5s
... Verify Default Meter Present in ONOS ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${ONOS_SSH_IP}
... ${ONOS_SSH_PORT} ${of_id} ${onu_port}
# Bring up the device and verify it authenticates
Wait Until Keyword Succeeds ${timeout} 5s Validate Device ENABLED ACTIVE REACHABLE
... ${onu_device_id} onu=True onu_reason=omci-flows-pushed by_dev_id=True
${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
... /tmp/wpa ${src['dp_iface_name']} log
Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
... ${src['container_type']} ${src['container_name']} ${wpa_log}
Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
... ${ONOS_SSH_PORT} ${onu_port}
END
${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
Log ${podStatusOutput}
${countBeforeUpgrade}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
${num_comps_under_test}= Get Length ${list_voltha_comps_under_test}
FOR ${I} IN RANGE 0 ${num_comps_under_test}
${label}= Set Variable ${list_voltha_comps_under_test}[${I}][label]
${container}= Set Variable ${list_voltha_comps_under_test}[${I}][container]
${image}= Set Variable ${list_voltha_comps_under_test}[${I}][image]
${pod_image} ${app_ver} ${helm_chart} Get Pod Image And App Version And Helm Chart By Label
... ${NAMESPACE} app ${label}
Log ${label}: image, app ver & helm chart before upgrade: ${pod_image}, ${app_ver} & ${helm_chart}
${deployment}= Wait Until Keyword Succeeds ${timeout} 15s
... Get K8s Deployment by Pod Label ${NAMESPACE} app ${label}
Wait Until Keyword Succeeds ${timeout} 15s Deploy Pod New Image ${NAMESPACE} ${deployment}
... ${container} ${image}
# Static sleep to let image-update progress before initiating subscriber-add system-operation
Sleep 10s
Provision Subscribers
Wait Until Keyword Succeeds ${timeout} 3s Validate Pods Status By Label ${NAMESPACE}
... app ${label} Running
Wait Until Keyword Succeeds ${timeout} 3s Pods Are Ready By Label ${NAMESPACE} app ${label}
Wait Until Keyword Succeeds ${timeout} 3s Verify Pod Image ${NAMESPACE} app ${label} ${image}
${pod_image_1} ${app_ver_1} ${helm_chart_1} Get Pod Image And App Version And Helm Chart By Label
... ${NAMESPACE} app ${label}
Log ${label}: image, app ver & helm chart after upgrade: ${pod_image_1}, ${app_ver_1} & ${helm_chart_1}
Restart VOLTHA Port Forward voltha-api
# Static sleep to let voltctl tcp connection establish
Sleep 5s
Verify Provisioned Subscribers
Unprovision Subscribers
END
${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
Log ${podStatusOutput}
${countAfterUpgrade}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
Should Be Equal As Strings ${countAfterUpgrade} ${countBeforeUpgrade}
# Additional Verification
Wait Until Keyword Succeeds ${timeout} 2s Delete All Devices and Verify
Setup
Run Keyword If ${has_dataplane} Clean Up Linux
Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
*** Keywords ***
Setup Suite
[Documentation] Set up the test suite
Common Test Suite Setup
Create Voltha Comp Under Test List
Teardown Suite
[Documentation] Tear down steps for the suite
Run Keyword If ${has_dataplane} Clean Up Linux
Close All ONOS SSH Connections
Create Voltha Comp Under Test List
[Documentation] Creates a list of Voltha Components to Test from the input variable string
... The input string is expected to be in format:
... <comp-label>,<comp-container>,<comp-image>*<comp-label>,<comp-container>,<comp-image>*
${list_voltha_comps_under_test} Create List
@{comps_under_test_arr}= Split String ${voltha_comps_under_test} *
${num_comps_under_test}= Get Length ${comps_under_test_arr}
FOR ${I} IN RANGE 0 ${num_comps_under_test}-1
@{comp_under_test_arr}= Split String ${comps_under_test_arr[${I}]} ,
${label}= Set Variable ${comp_under_test_arr[0]}
${container}= Set Variable ${comp_under_test_arr[1]}
${image}= Set Variable ${comp_under_test_arr[2]}
${comp_under_test} Create Dictionary label ${label} container ${container} image ${image}
Append To List ${list_voltha_comps_under_test} ${comp_under_test}
END
Log ${list_voltha_comps_under_test}
Set Suite Variable ${list_voltha_comps_under_test}
Provision Subscribers
[Documentation] This keyword provisions/adds all the subscribers on all the devices
FOR ${I} IN RANGE 0 ${num_all_onus}
${src}= Set Variable ${hosts.src[${I}]}
${dst}= Set Variable ${hosts.dst[${I}]}
${of_id}= Get ofID From OLT List ${src['olt']}
${onu_device_id}= Get Device ID From SN ${src['onu']}
${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
... ${of_id} ${src['uni_id']}
Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command use single connection ${ONOS_SSH_IP}
... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
END
Verify Provisioned Subscribers
[Documentation] This keyword verifies all the subscribers on all the devices
FOR ${I} IN RANGE 0 ${num_all_onus}
${src}= Set Variable ${hosts.src[${I}]}
${dst}= Set Variable ${hosts.dst[${I}]}
${of_id}= Get ofID From OLT List ${src['olt']}
${nni_port}= Wait Until Keyword Succeeds ${timeout} 2s Get NNI Port in ONOS ${of_id}
${onu_device_id}= Get Device ID From SN ${src['onu']}
${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
... ${of_id} ${src['uni_id']}
Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
# Verify subscriber access flows are added for the ONU port
Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
... Wait Until Keyword Succeeds ${timeout} 2s
... Check Ping True ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
END
Unprovision Subscribers
[Documentation] This keyword unprovisions/deletes all the subscribers on all the devices
FOR ${I} IN RANGE 0 ${num_all_onus}
${src}= Set Variable ${hosts.src[${I}]}
${dst}= Set Variable ${hosts.dst[${I}]}
${of_id}= Get ofID From OLT List ${src['olt']}
${nni_port}= Wait Until Keyword Succeeds ${timeout} 2s Get NNI Port in ONOS ${of_id}
${onu_device_id}= Get Device ID From SN ${src['onu']}
${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
... ${of_id} ${src['uni_id']}
Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command use single connection ${ONOS_SSH_IP}
... ${ONOS_SSH_PORT} volt-remove-subscriber-access ${of_id} ${onu_port}
Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
... Wait Until Keyword Succeeds ${timeout} 2s
... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
... ELSE Sleep 15s
END