Build virtualenv using the Makefile
Add tidy target (don't use yet as there's a bug)
Make lint target cover all the files
Start at documentation generation using libdoc/testdoc within robot
Change-Id: I8117baf4bd006588ae9383e0731c49e17102348c
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..ad8b8eb
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,7 @@
+# python virtualenv
+vst_venv
+
+# robot framework output
+output.xml
+report.xml
+log.html
diff --git a/Makefile b/Makefile
index 8aacf10..b99d7ce 100644
--- a/Makefile
+++ b/Makefile
@@ -12,8 +12,55 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-LINT_ARGS ?= --verbose --configure LineTooLong:130 --configure TooManyTestSteps:15 --ignore TooFewTestSteps --ignore TooFewKeywordSteps
+# use bash for pushd/popd, and to fail quickly. virtualenv's activate
+# has undefined variables, so no -u
+SHELL = bash -e -o pipefail
+# Variables
+LINT_ARGS ?= --verbose --configure LineTooLong:120 --configure TooManyTestSteps:15
+VERSION ?= $(shell cat ./VERSION)
-lint:
- rflint $(LINT_ARGS) .
+sanity-kind: ROBOT_PORT_ARGS ?= -v ONOS_REST_PORT:8181 -v ONOS_SSH_PORT:8101
+sanity-kind: ROBOT_TEST_ARGS ?= --exclude notready --critical sanity
+sanity-kind: ROBOT_MISC_ARGS ?= -v num_onus:1
+sanity-kind: sanity
+
+# virtualenv for the robot tools
+vst_venv:
+ virtualenv $@ ;\
+ source ./$@/bin/activate ;\
+ pip install -r requirements.txt
+
+test: lint
+
+lint: vst_venv
+ source ./vst_venv/bin/activate ;\
+ set -u ;\
+ find . -name *.robot -exec rflint $(LINT_ARGS) {} +
+
+# tidy will be more useful once this issue with removing leading comments is
+# resolved: https://github.com/robotframework/robotframework/issues/3263
+tidy:
+ source ./vst_venv/bin/activate ;\
+ set -u ;\
+ find . -name *.robot -exec python -m robot.tidy --inplace {} \;
+
+sanity: vst_venv
+ source ./vst_venv/bin/activate ;\
+ set -u ;\
+ cd tests/sanity ;\
+ robot $(ROBOT_PORT_ARGS) $(ROBOT_TEST_ARGS) $(ROBOT_MISC_ARGS) sanity.robot
+
+gendocs: vst_venv
+ source ./vst_venv/bin/activate ;\
+ set -u ;\
+ mkdir -p gendocs ;\
+ python -m robot.libdoc --format HTML libraries/onos.robot gendocs/lib_onos_robot.html ;\
+ python -m robot.testdoc tests/Voltha_PODTests.robot gendocs/voltha_podtests.html
+
+# explore use of --docformat REST - integration w/Sphinx?
+clean:
+ find . -name output.xml -print
+
+clean-all: clean
+ rm -rf vst_venv gendocs
diff --git a/README.md b/README.md
index b59c455..76e51ee 100644
--- a/README.md
+++ b/README.md
@@ -1,23 +1,31 @@
-# System-Tests
+# VOLTHA System Tests
-Automated test-suites to validate the stability/functionality of VOLTHA. Tests that reside in here should be written in Robot Framework and Python.
+Automated test-suites to validate the stability/functionality of VOLTHA. Tests
+that reside in here should be written in Robot Framework and Python.
Intended use includes:
-* Sanity testing
-* Regression testing
-* Acceptance testing
* Functional testing
+* Integration and Acceptance testing
+* Sanity and Regression testing
* Scale Testing using BBSIM
-* Failure/Scenario testing
+* Failure scenario testing
## Prerequisites
-* Python Virtual-Env
-* `voltctl` - a command line tool to access VOLTHA. Reference - [voltctl](https://github.com/ciena/voltctl)
-* `kubectl` - a command line tool to access your Kubernetes Clusers. Reference - [kubectl](https://kubernetes.io/docs/reference/kubectl/kubectl/)
- * `voltctl` and `kubectl` should be configured to your system under test prior to any test executions
-Directory Structures are as followed:
+* Python [virtualenv](https://virtualenv.pypa.io/en/latest/)
+
+* `voltctl` - a command line tool to access VOLTHA. Reference -
+ [voltctl](https://github.com/opencord/voltctl)
+
+* `kubectl` - a command line tool to access your Kubernetes Clusers. Reference
+ - [kubectl](https://kubernetes.io/docs/reference/kubectl/kubectl/)
+
+* `voltctl` and `kubectl` must be properly configured on your system
+ prior to any test executions
+
+Directory is structured as follows:
+
```
├── tests
└── sanity/ // basic tests that should always pass. Will be used as gating-patchsets
@@ -26,19 +34,44 @@
└── variables // shared variables across various test suites
```
+## Running the sanity tests
-## Getting Started
-1. Download `voltha-system-tests`
- * `git clone https://gerrit.opencord.org/voltha-system-tests`
+To run the the sanity tests using an environment previously
+set up by [kind-voltha](https://github.com/ciena/kind-voltha), run:
+```
+make sanity-kind
+```
-2. Create test virtual-environment
- * `cd voltha-system-tests/`
- * `source setup_venv.sh`
-3. Running Test-Suites
- * Navigate to desired test suite location
- * `robot --exclude notready sanity.robot`
+This test execution will generate three report files in
+`voltha-system-tests/tests/sanity` (`output.xml`,
+`report.html`, `log.html`). View the `report.html` page in a browser
+to analyze the results.
-This test execution will generate three report files (`output.xml`, `report.html`, `log.html`). View the `report.html` page to analyze the results.
+## Test variables
+
+The `make sanity-kind` target is equivalent to the following:
+```
+ROBOT_PORT_ARGS="-v ONOS_REST_PORT:8181 -v ONOS_SSH_PORT:8101" \
+ROBOT_TEST_ARGS="--exclude notready --critical sanity" \
+ROBOT_MISC_ARGS="-v num_onus:1" \
+make sanity
+```
+If you are running the tests in another environment, you can run `make sanity`
+with the arguments appropriate for your environment. Look at
+[variables.robot](variables/variables.robot) for a list of variables that
+you may need to override.
+
+## Adding to the tests
+
+Most additions should be done by adding keywords to the libraries, then
+calling these keywords from the tests. Consult a
+[guide on how to write good Robot framework tests](https://github.com/robotframework/HowToWriteGoodTestCases/blob/master/HowToWriteGoodTestCases.rst).
+
+Make sure that `make lint` passes, which runs
+[robotframework-lint](https://github.com/boakley/robotframework-lint) on any
+new code that is created.
## WIP:
-* Containerizing test environment so these tests can be run independent of the system
\ No newline at end of file
+
+* Containerizing test environment so these tests can be run independent of the system.
+
diff --git a/libraries/onos.robot b/libraries/onos.robot
index 9f7c802..9250ab8 100644
--- a/libraries/onos.robot
+++ b/libraries/onos.robot
@@ -37,35 +37,37 @@
[Return] ${output}
Validate OLT Device in ONOS
- [Documentation] Checks if olt has been connected to ONOS
[Arguments] ${serial_number}
+ [Documentation] Checks if olt has been connected to ONOS
${resp}= Get Request ONOS onos/v1/devices
${jsondata}= To Json ${resp.content}
Should Not Be Empty ${jsondata['devices']}
${length}= Get Length ${jsondata['devices']}
@{serial_numbers}= Create List
- : FOR ${INDEX} IN RANGE 0 ${length}
- \ ${value}= Get From List ${jsondata['devices']} ${INDEX}
- \ ${of_id}= Get From Dictionary ${value} id
- \ ${sn}= Get From Dictionary ${value} serial
- \ Run Keyword If '${sn}' == '${serial_number}' Exit For Loop
+ FOR ${INDEX} IN RANGE 0 ${length}
+ ${value}= Get From List ${jsondata['devices']} ${INDEX}
+ ${of_id}= Get From Dictionary ${value} id
+ ${sn}= Get From Dictionary ${value} serial
+ Run Keyword If '${sn}' == '${serial_number}' Exit For Loop
+ END
Should Be Equal As Strings ${sn} ${serial_number}
[Return] ${of_id}
Get ONU Port in ONOS
+ [Arguments] ${onu_serial_number} ${olt_of_id}
[Documentation] Retrieves ONU port for the ONU in ONOS
- [Arguments] ${onu_serial_number} ${olt_of_id}
${resp}= Get Request ONOS onos/v1/devices/${olt_of_id}/ports
${jsondata}= To Json ${resp.content}
Should Not Be Empty ${jsondata['ports']}
${length}= Get Length ${jsondata['ports']}
@{ports}= Create List
- : FOR ${INDEX} IN RANGE 0 ${length}
- \ ${value}= Get From List ${jsondata['ports']} ${INDEX}
- \ ${annotations}= Get From Dictionary ${value} annotations
- \ ${onu_port}= Get From Dictionary ${value} port
- \ ${portName}= Get From Dictionary ${annotations} portName
- \ Run Keyword If '${portName}' == '${onu_serial_number}' Exit For Loop
+ FOR ${INDEX} IN RANGE 0 ${length}
+ ${value}= Get From List ${jsondata['ports']} ${INDEX}
+ ${annotations}= Get From Dictionary ${value} annotations
+ ${onu_port}= Get From Dictionary ${value} port
+ ${portName}= Get From Dictionary ${annotations} portName
+ Run Keyword If '${portName}' == '${onu_serial_number}' Exit For Loop
+ END
Should Be Equal As Strings ${portName} ${onu_serial_number}
[Return] ${onu_port}
diff --git a/libraries/utils.robot b/libraries/utils.robot
index db67643..e303379 100644
--- a/libraries/utils.robot
+++ b/libraries/utils.robot
@@ -31,5 +31,4 @@
# check voltctl and kubectl configured
${voltctl_rc}= Run And Return RC ${VOLTCTL_CONFIG}; voltctl device list
${kubectl_rc}= Run And Return RC ${KUBECTL_CONFIG}; kubectl get pods
- Run Keyword If ${voltctl_rc} != 0 or ${kubectl_rc} != 0 FATAL ERROR
- ... VOLTCTL and KUBECTL not configured. Please configure before executing tests.
+ Run Keyword If ${voltctl_rc} != 0 or ${kubectl_rc} != 0 FATAL ERROR VOLTCTL and KUBECTL not configured. Please configure before executing tests.
diff --git a/libraries/voltctl.robot b/libraries/voltctl.robot
index d7dc48a..88541c7 100644
--- a/libraries/voltctl.robot
+++ b/libraries/voltctl.robot
@@ -30,24 +30,26 @@
[Arguments] ${ip} ${port}
[Documentation] Parses the output of "voltctl device list" and inspects device ${serial_number}
#create/preprovision device
- ${rc} ${device_id}= Run and Return Rc and Output
- ... ${VOLTCTL_CONFIG}; voltctl device create -t openolt -H ${ip}:${port}
+ ${rc} ${device_id}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl device create -t openolt -H ${ip}:${port}
Should Be Equal As Integers ${rc} 0
[Return] ${device_id}
Enable Device
[Arguments] ${device_id}
+ [Documentation] Enables a device in VOLTHA
${rc} ${output}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl device enable ${device_id}
Should Be Equal As Integers ${rc} 0
Get Device Flows from Voltha
[Arguments] ${device_id}
+ [Documentation] Gets device flows from VOLTHA
${rc} ${output}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl device flows ${device_id}
Should Be Equal As Integers ${rc} 0
[Return] ${output}
Get Logical Device Output from Voltha
[Arguments] ${device_id}
+ [Documentation] Gets logicaldevice flows and ports from VOLTHA
${rc1} ${flows}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl logicaldevice flows ${device_id}
${rc2} ${ports}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl logicaldevice ports ${device_id}
Log ${flows}
@@ -57,6 +59,7 @@
Get Device Output from Voltha
[Arguments] ${device_id}
+ [Documentation] Gets device flows and ports from VOLTHA
${rc1} ${flows}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl device flows ${device_id}
${rc2} ${ports}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl device ports ${device_id}
Log ${flows}
@@ -72,20 +75,20 @@
${jsondata}= To Json ${output}
Log ${jsondata}
${length}= Get Length ${jsondata}
- : FOR ${INDEX} IN RANGE 0 ${length}
- \ ${value}= Get From List ${jsondata} ${INDEX}
- \ ${astate}= Get From Dictionary ${value} adminstate
- \ ${opstatus}= Get From Dictionary ${value} operstatus
- \ ${cstatus}= Get From Dictionary ${value} connectstatus
- \ ${sn}= Get From Dictionary ${value} serialnumber
- \ ${mib_state}= Get From Dictionary ${value} reason
- \ Run Keyword If '${sn}' == '${serial_number}' Exit For Loop
+ FOR ${INDEX} IN RANGE 0 ${length}
+ ${value}= Get From List ${jsondata} ${INDEX}
+ ${astate}= Get From Dictionary ${value} adminstate
+ ${opstatus}= Get From Dictionary ${value} operstatus
+ ${cstatus}= Get From Dictionary ${value} connectstatus
+ ${sn}= Get From Dictionary ${value} serialnumber
+ ${mib_state}= Get From Dictionary ${value} reason
+ Run Keyword If '${sn}' == '${serial_number}' Exit For Loop
+ END
Should Be Equal ${astate} ${admin_state} Device ${serial_number} admin_state != ENABLED values=False
Should Be Equal ${opstatus} ${oper_status} Device ${serial_number} oper_status != ACTIVE values=False
Should Be Equal ${cstatus} ${connect_status} Device ${serial_number} connect_status != REACHABLE values=False
Run Keyword If '${onu}' == 'True' Should Be Equal ${mib_state} ${onu_reason} Device ${serial_number} mib_state incorrect values=False
-
Get Device ID From SN
[Arguments] ${serial_number}
[Documentation] Gets the device id by matching for ${serial_number}
@@ -93,11 +96,12 @@
${jsondata}= To Json ${output}
Log ${jsondata}
${length}= Get Length ${jsondata}
- : FOR ${INDEX} IN RANGE 0 ${length}
- \ ${value}= Get From List ${jsondata} ${INDEX}
- \ ${id}= Get From Dictionary ${value} id
- \ ${sn}= Get From Dictionary ${value} serialnumber
- \ Run Keyword If '${sn}' == '${serial_number}' Exit For Loop
+ FOR ${INDEX} IN RANGE 0 ${length}
+ ${value}= Get From List ${jsondata} ${INDEX}
+ ${id}= Get From Dictionary ${value} id
+ ${sn}= Get From Dictionary ${value} serialnumber
+ Run Keyword If '${sn}' == '${serial_number}' Exit For Loop
+ END
[Return] ${id}
Get Logical Device ID From SN
@@ -107,11 +111,12 @@
${jsondata}= To Json ${output}
Log ${jsondata}
${length}= Get Length ${jsondata}
- : FOR ${INDEX} IN RANGE 0 ${length}
- \ ${value}= Get From List ${jsondata} ${INDEX}
- \ ${id}= Get From Dictionary ${value} parentid
- \ ${sn}= Get From Dictionary ${value} serialnumber
- \ Run Keyword If '${sn}' == '${serial_number}' Exit For Loop
+ FOR ${INDEX} IN RANGE 0 ${length}
+ ${value}= Get From List ${jsondata} ${INDEX}
+ ${id}= Get From Dictionary ${value} parentid
+ ${sn}= Get From Dictionary ${value} serialnumber
+ Run Keyword If '${sn}' == '${serial_number}' Exit For Loop
+ END
[Return] ${id}
Validate Device Removed
@@ -122,8 +127,9 @@
Log ${jsondata}
${length}= Get Length ${jsondata}
@{ids}= Create List
- : FOR ${INDEX} IN RANGE 0 ${length}
- \ ${value}= Get From List ${jsondata} ${INDEX}
- \ ${device_id}= Get From Dictionary ${value} id
- \ Append To List ${ids} ${device_id}
+ FOR ${INDEX} IN RANGE 0 ${length}
+ ${value}= Get From List ${jsondata} ${INDEX}
+ ${device_id}= Get From Dictionary ${value} id
+ Append To List ${ids} ${device_id}
+ END
List Should Not Contain Value ${ids} ${id}
diff --git a/requirements.txt b/requirements.txt
new file mode 100644
index 0000000..924f78b
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1,14 @@
+cryptography==2.4.2
+docutils
+paramiko==2.4.2
+pexpect
+psycopg2==2.7.7
+pygments
+pyyaml
+robotframework
+robotframework-databaselibrary
+robotframework-httplibrary
+robotframework-kafkalibrary
+robotframework-lint
+robotframework-requests
+robotframework-sshlibrary
diff --git a/setup_venv.sh b/setup_venv.sh
deleted file mode 100644
index 523e47d..0000000
--- a/setup_venv.sh
+++ /dev/null
@@ -1,36 +0,0 @@
-#!/usr/bin/env bash
-
-# Copyright 2017-present Open Networking Foundation
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# sets up a python virtualenv for running voltha system tests
-
-WORKSPACE=${WORKSPACE:-.}
-VENVDIR="${WORKSPACE}/venv-voltha-tests"
-
-# create venv if it's not yet there
-if [ ! -x "${VENVDIR}/bin/activate" ]; then
- echo "Setting up dev/test virtualenv in ${VENVDIR} for VOLTHA-System-Tests"
- virtualenv -q "${VENVDIR}" --no-site-packages
- echo "Virtualenv created."
-fi
-
-echo "Installing python requirements in virtualenv with pip"
-source "${VENVDIR}/bin/activate"
-pip install cryptography==2.4.2 robotframework robotframework-requests robotframework-sshlibrary \
- pexpect robotframework-httplibrary robotframework-kafkalibrary pygments pyyaml \
- robotframework-databaselibrary psycopg2==2.7.7 paramiko==2.4.2
-pip install requests tinydb
-
-echo "VOLTHA-System-Tests virtualenv created. Run 'source ${VENVDIR}/bin/activate'."
diff --git a/tests/Voltha_PODTests.robot b/tests/Voltha_PODTests.robot
index f55afc0..6d1949d 100644
--- a/tests/Voltha_PODTests.robot
+++ b/tests/Voltha_PODTests.robot
@@ -13,7 +13,7 @@
# limitations under the License.
*** Settings ***
-Documentation Test various end-to-end scenarios
+Documentation Test various end-to-end scenarios
Suite Setup Setup Suite
Test Setup Setup
Test Teardown Teardown
@@ -33,38 +33,41 @@
Resource ../variables/variables.robot
*** Variables ***
-${POD_NAME} flex-ocp-cord
-${KUBERNETES_CONF} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
-${KUBERNETES_CONFIGS_DIR} ~/pod-configs/kubernetes-configs
-#${KUBERNETES_CONFIGS_DIR} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
-${KUBERNETES_YAML} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.yml
-${HELM_CHARTS_DIR} ~/helm-charts
-${VOLTHA_POD_NUM} 8
-${timeout} 90s
-${num_onus} 1
-${of_id} 0
-${logical_id} 0
+${POD_NAME} flex-ocp-cord
+${KUBERNETES_CONF} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
+${KUBERNETES_CONFIGS_DIR} ~/pod-configs/kubernetes-configs
+#${KUBERNETES_CONFIGS_DIR} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
+${KUBERNETES_YAML} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.yml
+${HELM_CHARTS_DIR} ~/helm-charts
+${VOLTHA_POD_NUM} 8
+${timeout} 90s
+${num_onus} 1
+${of_id} 0
+${logical_id} 0
*** Test Cases ***
Sanity E2E Test for OLT/ONU on POD
[Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
... Validate successful authentication/DHCP/E2E ping for the tech profile that is used
- #[Setup] Clean Up Linux
[Tags] test1
- ${of_id}= Wait Until Keyword Succeeds 60s 15s Validate OLT Device in ONOS ${olt_serial_number}
- #Wait Until Keyword Succeeds 60s 2s Verify Eapol Flows Added ${k8s_node_ip} ${ONOS_SSH_PORT} 5
- Validate Authentication True ${src0['dp_iface_name']} wpa_supplicant.conf ${src0['ip']} ${src0['user']} ${src0['pass']} ${src0['container_type']} ${src0['container_name']}
+ #[Setup] Clean Up Linux
+ ${of_id}= Wait Until Keyword Succeeds 60s 15s Validate OLT Device in ONOS ${olt_serial_number}
+ #Wait Until Keyword Succeeds 60s 2s Verify Eapol Flows Added ${k8s_node_ip} ${ONOS_SSH_PORT} 5
+ Validate Authentication True ${src0['dp_iface_name']} wpa_supplicant.conf ${src0['ip']} ${src0['user']} ${src0['pass']}
+ ... ${src0['container_type']} ${src0['container_name']}
#Validate ONU authenticated in ONOS
Wait Until Keyword Succeeds 90s 2s Verify Number of AAA-Users ${k8s_node_ip} ${ONOS_SSH_PORT} ${num_onus}
Wait Until Keyword Succeeds 60s 2s Execute ONOS Command ${k8s_node_ip} ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} 16
# Perform dhclient and ping operations
- Validate DHCP and Ping True True ${src0['dp_iface_name']} ${src0['s_tag']} ${src0['c_tag']} ${dst0['dp_iface_ip_qinq']} ${src0['ip']} ${src0['user']} ${src0['pass']} ${src0['container_type']} ${src0['container_name']} ${dst0['dp_iface_name']} ${dst0['ip']} ${dst0['user']} ${dst0['pass']} ${dst0['container_type']} ${dst0['container_name']}
+ Validate DHCP and Ping True True ${src0['dp_iface_name']} ${src0['s_tag']} ${src0['c_tag']} ${dst0['dp_iface_ip_qinq']}
+ ... ${src0['ip']} ${src0['user']} ${src0['pass']} ${src0['container_type']} ${src0['container_name']} ${dst0['dp_iface_name']}
+ ... ${dst0['ip']} ${dst0['user']} ${dst0['pass']} ${dst0['container_type']} ${dst0['container_name']}
#Validate DHCP allocation in ONOS
- Wait Until Keyword Succeeds 60s 2s Validate DHCP Allocations ${k8s_node_ip} ${ONOS_SSH_PORT} ${num_onus}
-
+ Wait Until Keyword Succeeds 60s 2s Validate DHCP Allocations ${k8s_node_ip} ${ONOS_SSH_PORT} ${num_onus}
*** Keywords ***
Setup Suite
+ [Documentation] Setup the whole test suite
Set Global Variable ${KUBECTL_CONFIG} export KUBECONFIG=%{KUBECONFIG}
Set Global Variable ${export_kubeconfig} export KUBECONFIG=${KUBERNETES_CONF}
Set Global Variable ${of_id}
@@ -104,15 +107,18 @@
Set Suite Variable ${datetime}
Setup
+ [Documentation] Pre-test Setup
#create/preprovision device
${olt_device_id}= Create Device ${olt_ip} ${OLT_PORT}
Set Suite Variable ${olt_device_id}
#enable device
Enable Device ${olt_device_id}
#validate olt states
- Wait Until Keyword Succeeds 60s 5s Validate Device ${olt_serial_number} ENABLED ACTIVE REACHABLE
+ Wait Until Keyword Succeeds 60s 5s Validate Device ${olt_serial_number} ENABLED ACTIVE
+ ... REACHABLE
#validate onu states
- Wait Until Keyword Succeeds 60s 5s Validate Device ${onu_serial_number} ENABLED ACTIVE REACHABLE onu=True onu_reason=tech-profile-config-download-success
+ Wait Until Keyword Succeeds 60s 5s Validate Device ${onu_serial_number} ENABLED ACTIVE
+ ... REACHABLE onu=True onu_reason=tech-profile-config-download-success
#get onu device id
${onu_device_id}= Get Device ID From SN ${onu_serial_number}
Set Suite Variable ${onu_device_id}
@@ -129,9 +135,12 @@
Clean Up Linux
[Documentation] Kill processes and clean up interfaces on src+dst servers
- Run Keyword And Ignore Error Kill Linux Process [w]pa_supplicant ${src0['ip']} ${src0['user']} ${src0['pass']} ${src0['container_type']} ${src0['container_name']}
- Run Keyword And Ignore Error Kill Linux Process [d]hclient ${src0['ip']} ${src0['user']} ${src0['pass']} ${src0['container_type']} ${src0['container_name']}
- Run Keyword If '${dst0['ip']}' != '${None}' Run Keyword And Ignore Error Kill Linux Process [d]hcpd ${dst0['ip']} ${dst0['user']} ${dst0['pass']} ${dst0['container_type']} ${dst0['container_name']}
+ Run Keyword And Ignore Error Kill Linux Process [w]pa_supplicant ${src0['ip']} ${src0['user']} ${src0['pass']} ${src0['container_type']}
+ ... ${src0['container_name']}
+ Run Keyword And Ignore Error Kill Linux Process [d]hclient ${src0['ip']} ${src0['user']} ${src0['pass']} ${src0['container_type']}
+ ... ${src0['container_name']}
+ Run Keyword If '${dst0['ip']}' != '${None}' Run Keyword And Ignore Error Kill Linux Process [d]hcpd ${dst0['ip']} ${dst0['user']}
+ ... ${dst0['pass']} ${dst0['container_type']} ${dst0['container_name']}
Delete IP Addresses from Interface on Remote Host ${src0['dp_iface_name']} ${src0['ip']} ${src0['user']} ${src0['pass']} ${src0['container_type']} ${src0['container_name']}
- Run Keyword If '${dst0['ip']}' != '${None}' Delete Interface on Remote Host ${dst0['dp_iface_name']}.${src0['s_tag']} ${dst0['ip']} ${dst0['user']} ${dst0['pass']} ${dst0['container_type']} ${dst0['container_name']}
-
+ Run Keyword If '${dst0['ip']}' != '${None}' Delete Interface on Remote Host ${dst0['dp_iface_name']}.${src0['s_tag']} ${dst0['ip']} ${dst0['user']} ${dst0['pass']}
+ ... ${dst0['container_type']} ${dst0['container_name']}
diff --git a/tests/sanity/sanity.robot b/tests/sanity/sanity.robot
index 00324df..2fb7794 100644
--- a/tests/sanity/sanity.robot
+++ b/tests/sanity/sanity.robot
@@ -17,19 +17,19 @@
Documentation Creates bbsim olt/onu and validates activataion
... Assumes voltha-go, go-based onu/olt adapters, and bbsim are installed
... voltctl and kubectl should be configured prior to running these tests
+Suite Setup Setup
+Suite Teardown Teardown
+Test Teardown Execute ONOS CLI Command ${server_ip} ${ONOS_SSH_PORT} flows -s
Library OperatingSystem
Resource ${CURDIR}/../../libraries/onos.robot
Resource ${CURDIR}/../../libraries/voltctl.robot
Resource ${CURDIR}/../../libraries/utils.robot
Resource ${CURDIR}/../../variables/variables.robot
-Suite Setup Setup
-Suite Teardown Teardown
-Test Teardown Execute ONOS CLI Command ${server_ip} ${ONOS_SSH_PORT} flows -s
*** Variables ***
-${server_ip} localhost
-${timeout} 90s
-${num_onus} 1
+${server_ip} localhost
+${timeout} 90s
+${num_onus} 1
*** Test Cases ***
Activate Device BBSIM OLT/ONU
@@ -43,9 +43,11 @@
#enable device
Enable Device ${olt_device_id}
#validate olt states
- Wait Until Keyword Succeeds 60s 5s Validate Device ${BBSIM_OLT_SN} ENABLED ACTIVE REACHABLE
+ Wait Until Keyword Succeeds 60s 5s Validate Device ${BBSIM_OLT_SN} ENABLED ACTIVE
+ ... REACHABLE
#validate onu states
- Wait Until Keyword Succeeds 60s 5s Validate Device ${BBSIM_ONU_SN} ENABLED ACTIVE REACHABLE onu=True onu_reason=tech-profile-config-download-success
+ Wait Until Keyword Succeeds 60s 5s Validate Device ${BBSIM_ONU_SN} ENABLED ACTIVE
+ ... REACHABLE onu=True onu_reason=tech-profile-config-download-success
#get onu device id
${onu_device_id}= Get Device ID From SN ${BBSIM_ONU_SN}
Set Suite Variable ${onu_device_id}
@@ -71,16 +73,16 @@
Add Subscriber-Access in ONOS
[Documentation] Through the olt-app in ONOS, execute 'volt-add-subscriber-access' and validate IP Flows
[Tags] sanity
- ## TODO: this works fine with 1 onu, but with multiple onus, we need to ensure this is executes
- ... prior to to dhclient starting. possible start a process after first test case to just attempt
- ... "volt-add-subscriber-access" to all onus periodically?
+ ## TODO: this works fine with 1 onu, but with multiple onus, we need to ensure this is executes
+ ## prior to to dhclient starting. possible start a process after first test case to just attempt
+ ## "volt-add-subscriber-access" to all onus periodically?
${output}= Execute ONOS CLI Command ${server_ip} ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} 16
Log ${output}
Validate DHCP Assignment in ONOS
[Documentation] After IP Flows are pushed to the device, BBSIM will start a dhclient for the ONU.
[Tags] sanity
- Wait Until Keyword Succeeds 120s 15s Validate DHCP Allocations ${server_ip} ${ONOS_SSH_PORT} ${num_onus}
+ Wait Until Keyword Succeeds 120s 15s Validate DHCP Allocations ${server_ip} ${ONOS_SSH_PORT} ${num_onus}
Delete Device and Verify
[Documentation] Disable -> Delete devices via voltctl and verify its removed
@@ -88,14 +90,16 @@
#disable/delete onu
${rc} ${output}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl device disable ${onu_device_id}
Should Be Equal As Integers ${rc} 0
- Wait Until Keyword Succeeds 60s 5s Validate Device ${BBSIM_ONU_SN} DISABLED UNKNOWN REACHABLE
+ Wait Until Keyword Succeeds 60s 5s Validate Device ${BBSIM_ONU_SN} DISABLED UNKNOWN
+ ... REACHABLE
${rc} ${output}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl device delete ${onu_device_id}
Should Be Equal As Integers ${rc} 0
Wait Until Keyword Succeeds 60s 5s Validate Device Removed ${onu_device_id}
#disable/delete olt
${rc} ${output}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl device disable ${olt_device_id}
Should Be Equal As Integers ${rc} 0
- Wait Until Keyword Succeeds 60s 5s Validate Device ${BBSIM_OLT_SN} DISABLED UNKNOWN REACHABLE
+ Wait Until Keyword Succeeds 60s 5s Validate Device ${BBSIM_OLT_SN} DISABLED UNKNOWN
+ ... REACHABLE
${rc} ${output}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl device delete ${olt_device_id}
Should Be Equal As Integers ${rc} 0
Wait Until Keyword Succeeds 60s 5s Validate Device Removed ${olt_device_id}
diff --git a/variables/variables.robot b/variables/variables.robot
index cece945..6a78c6e 100644
--- a/variables/variables.robot
+++ b/variables/variables.robot
@@ -13,11 +13,11 @@
# limitations under the License.
*** Variables ***
-${BBSIM_IP} bbsim.voltha.svc
-${BBSIM_PORT} 50060
-${BBSIM_DEVICE_ID} of:0000626273696d76
-${BBSIM_OLT_SN} BBSIMOLT000
-${BBSIM_ONU_SN} BBSM00000001
-${ONOS_REST_PORT} 30120
-${ONOS_SSH_PORT} 30115
-${OLT_PORT} 9191
+${BBSIM_IP} bbsim.voltha.svc
+${BBSIM_PORT} 50060
+${BBSIM_DEVICE_ID} of:0000626273696d76
+${BBSIM_OLT_SN} BBSIMOLT000
+${BBSIM_ONU_SN} BBSM00000001
+${ONOS_REST_PORT} 30120
+${ONOS_SSH_PORT} 30115
+${OLT_PORT} 9191